Turing Award winner Richard Sutton argues that raw compute alone is insufficient for true intelligence and that the field of AI science remains far from finished.
In a recent post, the creator of the influential 2019 essay “The Bitter Lesson” stated that AI requires clever new methods rather than relying solely on computational power. His comments sparked significant discussion on X.
Sutton’s position aligns with his earlier criticisms of large language model scaling. He described the use of synthetic data as a “big mistake.” He believes any simulation of the real world is only “microscopic” because reality is infinitely complex. He argues that human expertise acts as a bottleneck preventing true scaling.
Instead, Sutton calls for agents equipped with world models that continuously learn from their own experience. He notes that today’s language models do not keep learning after training and that current systems fail to capture the complexity of the world found in internet text. What is needed is continuous learning that does not destroy previously acquired knowledge.
LLMs under scrutiny
Ai researcher Jerry Tworek, who left OpenAI to explore new approaches, places Sutton’s latest statement in context. He agrees the bitter lesson still holds but adds that as more compute goes into model training, the algorithm matters more.
Some algorithms scale while others hit dead ends. Many dimensions of scaling remain completely unexplored. Tworek writes that the temporary success of transformer scaling, combined with a misreading of the bitter lesson, “has reward-hacked the brains of a whole generation of scientists.”
AI researcher Jayden Teoh adds another angle. He says he has repeatedly seen the bitter lesson hold back academic research, with promising ideas dismissed for straying too far from its spirit. Teoh argues this is not Sutton’s fault but a misreading of his original message.
Teoh also argues that ideas which did not follow the bitter lesson were often replaced by cleaner, more general approaches. He says those ideas were not wrong; they often enabled progress at the time. He wants the research community to stop dismissing such work as soon as it appears.
“An idea doesn’t necessarily have to survive scaling forever to be an important contribution to AI progress,” Teoh writes.
The tension between Sutton’s call for new learning methods and Tworek’s criticism of blind transformer scaling appears in a growing wave of startups. In the latest Frontier Radar #5, we tracked 45 new AI labs betting on exactly these gaps.
Tworek runs Core Automation, one of those labs, searching for learning algorithms beyond pretraining and reinforcement learning. What unites these founders, including Sutton with Oak Lab, is that they believe in much more capable AI but doubt today’s recipe alone will get there.
Some still bet on LLM scaling
Prominent industry leaders see things very differently. OpenAI CEO Sam Altman considers large language model scaling far from exhausted. He accuses LLM critics of slowing down an entire generation of researchers by being too sure about what scaling cannot do.
As evidence, he points to an OpenAI model that recently disproved a mathematical conjecture mathematicians had long failed to crack. He considers world models useful, for instance in robotics, but says the data clearly favour further LLM scaling.
Anthropic CEO Dario Amodei argues similarly, calling scaling progress a rainbow with no end. As CEOs of the two AI companies whose entire technical approach rests on this bet, Amodei and Altman are obviously biased in its favour.
What it means
For developers building models or using APIs, the immediate impact is limited to research strategy. The source text does not state that customer products, account privacy, or API behaviour will change. The debate concerns internal model training and evaluation methods.
Users should note that the community is split on whether to pursue continuous learning agents or push scaling limits further. Both camps believe the current path is insufficient for long-term goals.
