Recorded 25 November 2025 · 96 min
Models generalize dramatically worse than people, and that, not a shortage of compute, is now the bottleneck. Ilya Sutskever argues the 2020-2025 age of scaling is over and the field is back in an age of research, only with very large computers.
Sutskever, co-founder of SSI, sits with an interviewer who presses on every soft spot. The talk runs from why models score well on hard evals but stumble on real work, to what human learning and emotions suggest about missing machine learning principles, to what a safe path to superintelligence looks like.
The tension is that he is confident about the diagnosis and openly withholds the cure. He says his ideas on human-like generalization cannot be discussed freely. He gives a 5 to 20 year timeline and bets that SSI's modest compute is enough to prove them.
Evals outrun economics — ▶ 1:32
Models look strong on hard evals while their economic impact lags dramatically behind, and Sutskever finds that hard to reconcile. The telltale case is vibe coding. Ask a model to fix a bug and it apologizes, fixes it and introduces a second bug. Flag the second and it brings back the first, and the two can alternate indefinitely. He does not claim to know why, only that something strange is going on. He still expects impact to land: AI will diffuse through the economy under very strong economic forces.
Researchers reward hack — ▶ 2:52
He offers two explanations for the eval gap. RL may make models too single-minded and unaware. Or, unlike pre-training, where the data answer was "everything", RL forces choices among a huge variety of environments. From what he hears, every company has teams producing new RL environments. He suspects people, inadvertently, take inspiration from the evals they want to look great at release. Combine that with inadequate generalization and it could explain much of the disconnect, which today we do not even understand.
Two students — ▶ 6:14
Student one practices competitive programming for 10,000 hours and memorizes every technique. Student two practices 100 hours and also does really well. Sutskever expects the second to do better in a career, because they have "it". Models are like the first student "but even more": every competitive programming problem ever, plus data augmentation. Deep preparation need not transfer. Pre-training he calls hard to reason about, the whole world projected onto text, with no real human analogue: after 15 years people know far less but know it more deeply.
Value functions and emotions — ▶ 11:26
A person with brain damage lost all emotional processing yet stayed articulate and passed puzzle tests. He took hours to choose socks and made bad financial decisions. Sutskever reads emotions as a hardcoded modulator of the human value function. Today's RL grades only the finished solution, so long tasks teach nothing until the end. A value function short-circuits that: after 1,000 steps down a dead end, the decision to take that path is penalized at once. He expects them to be used, but anything you can do with one, you can do without, just more slowly.
Back to research — ▶ 19:12
Sutskever dates 2012-2020 as the age of research and 2020-2025 as the age of scaling, error bars included. "Scaling" was a word that told everyone what to do, and pre-training was a low-risk recipe: more compute and data, near-guaranteed gains. Data is finite, though. RL is now the scaling target and, by what people say publicly, consumes more compute than pre-training, with long rollouts yielding little learning each. He will not call that scaling, only ask whether it is the most productive use of compute. He does not believe 100x more scale transforms everything.
Humans just learn better — ▶ 25:02
Models generalize dramatically worse than people, he says, and that is the most fundamental problem. Evolution could explain human skill at vision, hearing, locomotion and dexterity. He recalls recognizing cars well enough to drive at age 5. But language, math and coding are too recent for evolved priors, yet people are reliable, robust and fast there. His conclusion is that people simply have better machine learning, period: fewer samples, more unsupervised, and no verifiable reward needed. Two puzzles remain: sample efficiency, and teaching models without bespoke reward setups.
The undisclosed idea — ▶ 30:26
A teenager learns to drive in about 10 hours with no external teacher, because they have a value function that tells them immediately how badly they are driving. Human value judgment is also very robust, with few exceptions such as addiction. How should training be rebuilt to get this? Sutskever has a lot of opinions, but says not all machine learning ideas are discussed freely and this is one of them. Humans are his proof it can be done. One possible blocker: human neurons may do more compute than we think.
Research needs less compute — ▶ 36:40
Scaling "sucked out all the air in the room", leaving more companies than ideas. AlexNet ran on 2 GPUs. The transformer used 8 to 64 GPUs, and no experiment in that paper used more than 64 GPUs of 2017, about 2 of today's. The biggest compute still helps build the best system. SSI has raised $3 billion. He says rivals' larger sums are earmarked for inference and product staff. Challenged with $5-6 billion a year of OpenAI experiment spend, he says rivals have more workstreams, so their compute fragments. Revenue plan: focus on research.
Show the AI — ▶ 44:06
The case for a straight shot is avoiding the rat race and its trade-offs. The case against is that the world should see powerful AI. An essay about AI is incomparable to watching an AI do things: "Communicate the AI, not the idea." Even a straight-shot plan would release gradually, and the question is only what goes out the door first. SSI may change plan if timelines turn out long. His view has shifted toward more weight on incremental deployment in advance, because AGI is hard to imagine.
A learner, not finished — ▶ 47:29
AGI exists as a reaction to "narrow AI", and pre-training, which improves a model at everything roughly uniformly, reinforced it. Both overshot: a human is not an AGI, lacking vast knowledge and relying on continual learning. He pictures a superintelligent 15-year-old, eager and knowing little, deployed to learn on the job through trial and error. Instances learning different jobs and merging could be functionally superintelligent without recursive self-improvement. He expects rapid economic growth but not its speed. The world is big, regulation may intervene, and friendlier jurisdictions would grow faster.
Paranoia and capped power — ▶ 56:22
Because AI is hard to feel, he predicts behavior will change as it becomes more powerful. Fierce competitors will collaborate on safety, as OpenAI and Anthropic began to. Governments and the public will want to act. Labs will become much more paranoid. He argues the field fixated on self-improving AI because there are fewer ideas than companies, and proposes AI robustly aligned to care about sentient life. The interviewer notes most sentient beings will be AIs, and he allows it may not be the best criterion. He also wants the most powerful system capped, method unknown.
Long-run equilibrium — ▶ 1:08:32
If the first N dramatic systems care about humanity or sentient life, things go well for quite some time. The long run is harder. Universal high income is a short-term answer, and political structures have a shelf life. Personal AIs that earn money and advocate for each person leave the person no longer a participant, which he calls precarious. His one answer, which he says he does not like, is that people become part-AI through "Neuralink++", so understanding is transmitted wholesale and humans stay fully involved.
Evolution's social mystery — ▶ 1:11:36
Evolution can wire the desire for good smells through a chemical sensor. Social desires such as standing with others require the brain to assemble many signals, yet are reliably baked in and evolved recently. Sutskever knows of no good hypothesis. His speculation, a hardcoded brain location, is refuted by people born blind whose cortex is co-opted by another sense, and by children with half a brain removed whose regions all move to one hemisphere. Even people with strange mental conditions tend to care about social standing.
5 to 20 years — ▶ 1:22:23
His forecast is 5 to 20 years to a system that learns like a human and then becomes superhuman. Current approaches will go some distance and then peter out. They may earn stupendous revenue, though not profits, but will not be "it". He expects rivals to converge on alignment strategies. Past pattern says one lab advances, others scramble and prices fall, and competition loves specialization into niches such as litigation. The interviewer argues one learner could be instanced on every job. He calls that valid but trusts his strong intuition. SSI is squarely an age-of-research company.
Taste as top-down belief — ▶ 1:33:11
His research taste is an aesthetic of how AI should be, from thinking about people correctly. The artificial neuron, local learning rules, distributed representations and learning from experience all came from brain inspiration, filtered for what is fundamental. He looks for beauty, simplicity and elegance, with no room for ugliness. The more of these are present, the stronger the top-down belief. That belief is what tells you to keep debugging when experiments contradict you, because a result can be wrong because of a bug you cannot yet see.
Models generalize far worse than people, and no amount of added scale fixes that. Ilya Sutskever treats this as the most fundamental problem in AI today. It explains a gap he finds hard to make sense of: models score well on difficult evals while their economic impact lags well behind. The everyday symptom is familiar to anyone who codes with them. Ask a model to fix a bug and it introduces a second one. Point out the second and it brings back the first. It can alternate between the two indefinitely. Sutskever does not claim to know why. He offers two partial explanations, and both point at how the models are trained rather than how big they are.
The first is that reinforcement learning makes models too single-minded. The second is more pointed. Pre-training never forced anyone to choose data, because the answer was everything. RL does force the choice. Every lab has teams producing new RL environments, and the space of possible environments has enormous degrees of freedom. Sutskever suspects people pick them, inadvertently, by looking at the evals they want to ace at release. The human researchers become the reward hackers. Add weak generalization and you get a model that shines on benchmarks and stumbles in the field.
His analogy is two students. One practices competitive programming for 10,000 hours and memorizes every technique. The other practices 100 hours and has the "it" factor. The second does better in a career. Models are like the first student, only more so, because labs collect every competitive programming problem and augment the data for more. Preparation that deep does not obviously transfer.
If the problem is how models learn, then the last five years of strategy look different. Sutskever dates the age of research to 2012-2020 and the age of scaling to 2020-2025, error bars included. Scaling was powerful partly as a word. It told everyone what to do. It also gave companies a low-risk way to spend: mix compute and data into a network of a given size and you know you will get results. Research offers no such guarantee. But data is finite, and the recipe is running out. RL is the successor. By what people say publicly, it now consumes more compute than pre-training, because long rollouts yield little learning each. He declines to call that scaling. The real question is whether it is the most productive use of compute. And he does not believe that 100x more scale would transform everything. Scaling also "sucked out all the air in the room", leaving more companies than ideas.
The obvious objection is that people had a head start. Evolution gave humans strong priors for vision, hearing, locomotion and dexterity. A child learns to drive in 10 hours, but the child's vision was already excellent. Sutskever grants this for those skills. It does not work for language, math and coding. Those domains barely existed until recently, yet people show reliability, robustness and fast learning in them. So whatever makes people good learners is probably not a complicated prior. People simply have better machine learning, period. They need fewer samples, learn without a verifiable reward and are far more robust.
His best clue is the value function. A teenager at the wheel knows almost at once how badly they are driving, and corrects without an external teacher. Today's RL grades only the final answer and spreads that score across every step, so a long task teaches nothing until it ends. A value function short-circuits the wait. In chess, losing a piece tells you the move was bad without finishing the game. After 1,000 steps down an unpromising path, you can penalize the decision made 1,000 steps earlier. Sutskever expects value functions to be used. He is careful about their ceiling. Anything you can do with one, you can do without, just more slowly. They are an efficiency gain, not the missing principle.
Humans add a wrinkle. He cites a patient who lost all emotional processing. The man stayed articulate and passed tests, yet took hours to choose socks and made bad financial decisions. Emotions look like a hardcoded layer that makes human value functions work. They are simple and robust. How evolution hard-coded high-level social desires, which have no sensor the way smell does, he calls a mystery with no satisfying hypothesis.
What is the missing principle itself? Here Sutskever stops. He says he has many opinions on how to build it and that these ideas cannot be discussed freely. The existence of human learners is his proof it can be done. He adds one caveat: human neurons may do more compute than assumed, which would make the problem harder. This is the weak point of the argument. The diagnosis is public and the cure is not.
That matters for SSI's bet, which is that research does not need maximal compute. AlexNet ran on 2 GPUs. The transformer was built on 8 to 64 GPUs, and no experiment in that paper used more than 64 GPUs of 2017, roughly 2 of today's. Frontier systems do benefit from more compute, especially when everyone shares a paradigm. But proving an idea is cheaper. SSI has raised $3 billion. The large raises elsewhere are earmarked for inference and for the engineers and salespeople a product needs, so what is left for research is closer than it looks. The interviewer pushes back with public estimates that OpenAI spends $5-6 billion a year on experiments alone. Sutskever answers that rivals carry more workstreams and modalities, so their compute fragments. How SSI will make money he leaves open.
If the goal is a model that learns like a person, then superintelligence is not a finished mind. Sutskever argues that AGI began as a reaction to "narrow AI", and that pre-training, which improves a model at everything more or less uniformly, reinforced the idea. Both overshot. A human is not an AGI. A person lacks vast knowledge and relies on continual learning. The thing to build is a superintelligent 15-year-old: eager, knowing little, deployed to learn on the job. Deployment becomes a trial-and-error process. The interviewer adds the consequence. Instances learning different jobs and merging what they learn would be functionally superintelligent even without recursive self-improvement. Sutskever expects rapid economic growth, with the speed unknown. The world is large and moves at its own pace, regulation might intervene, and friendlier jurisdictions would grow faster.
This is also why his view of release has moved. A straight shot to superintelligence would insulate a lab from the rat race. But people cannot feel something that does not exist yet, much as the young cannot feel what it is to be old. Hence: "Communicate the AI, not the idea." Even a straight-shot plan would release gradually. The only question is what goes out the door first. He then makes predictions. Fierce competitors will collaborate on safety, as OpenAI and Anthropic began to. Governments and the public will want to act. Once AI feels powerful, labs will become much more paranoid. A better target than self-improving AI, he says, is one robustly aligned to care about sentient life, with the power of the strongest system capped. He has no method for the cap.
The strongest objections land here. The interviewer notes most sentient beings will eventually be AIs, so human control may not be the right criterion. Sutskever allows it may not be the best. His own long-run answer for keeping humans in the loop is one he says he dislikes: people becoming part-AI through "Neuralink++", since personal AI agents that earn and advocate on our behalf would leave us as spectators.
The structural objection is sharper. If one lab gets a human-like continual learner first and can put an instance on every job, why would the gains spread? Sutskever points to market history. One company advances, others scramble, and prices fall. Competition also loves specialization, so firms would settle into niches, one strong at litigation, another elsewhere, each having sunk compute and experience into its own. The interviewer replies that this assumes many labs reach the learner together. Sutskever calls the argument valid and holds his ground on intuition: "In theory, there is no difference between theory and practice. In practice, there is."
He puts the arrival of a system that learns like a human and then turns superhuman at 5 to 20 years. Current approaches, he thinks, will earn stupendous revenue and then peter out. His confidence in the alternative rests on taste: a top-down belief built from beauty, simplicity and correct inspiration from the brain, which keeps you debugging when experiments contradict you. Whether that belief is right, no outsider can yet check.
The open question. The central claim is that human learning proves a machine learning principle exists that would fix generalization, and that SSI is on its trail. The evidence offered is an analogy to teenage drivers, and the idea itself is withheld. His prediction that rival labs will converge on alignment strategies, and that competition will spread the gains through specialization, rests on what he calls a strong intuition after the interviewer showed a first-mover case against it. Watch whether SSI shows any result on generalization, and whether a lab with a working continual learner ends up with a lead that does not erode.
Watch the full video at https://www.youtube.com/watch?v=aR20FWCCjAs.
Comments ()