Researchers put ASI-level AI research automation at 2-10 years out

Three frontier-adjacent researchers walk through why AI progress hasn't produced explosive recursive self-improvement yet, and disagree sharply on how far away it is. Their bets range from AI research being automated within 2 years to full superintelligence across all cognitive work taking 5-10 years, undercutting both doomer and skeptic narratives.

• Why 2036 could be normal — Beren Millidge says the most likely reason 2036 isn't a superintelligent world is a Moravec's-paradox-style failure of generalization, but calls this unlikely since RL already shows generalization in practice.
• Moore's Law analogy — Millidge compares the RL scaling paradigm to Moore's Law, noting pre-training loss hit diminishing returns before RL scaling gave a 'new curve,' and questions whether the current transformer-plus-RL recipe can discover the next such discontinuity.
• Chess Elo crossover — Dwarkesh Patel raises Ryan Greenblatt's point that chess bot Elo scores rose linearly for decades before a sharp discontinuity around human expert level, suggesting today's unimpressive real-world AI impact reflects models still climbing toward the human range rather than an asymptote.
• John Schulman on 2012 — Schulman recalls that at early OpenAI he expected minimizing log loss alone wouldn't reach intelligence because 'important bits' were a small fraction of the loss — yet next-token prediction worked anyway, illustrating unpredictable generalization.
• Kaplan scaling law bug — Millidge estimates that if an AI had caught the flawed intermediate-checkpoint methodology in the original Kaplan scaling laws (missing the annealing effect), it could have saved one to two years of research progress, implying roughly a 10x speedup on well-specified objectives.
• Distillation vs. centralization — Schulman argues distillation is the main force preventing model-provider consolidation, since anything learnable through RL is a small number of bits easily copied from trajectories.
• Chinese router services — Millidge says Chinese firms reportedly use proxy/router services that let mainland users access blocked US frontier models, and these services collect and sell the resulting prompt data as a distillation goldmine.
• Sonnet 5 vs GLM-5.3 — O'Neill notes Sonnet 5 and Opus 5 seem objectively worse than GLM-5.3 and Kimi K3 despite Anthropic's access to harder RL environments and logit distillation, suggesting realistic deployment data may matter more than environment difficulty.
• RL learns one bit — Schulman's earlier research argued RL teaches a model roughly one bit per episode (right or wrong), and Millidge explains RL's outsized effect as coming mostly from strong mid-training warm starts that get models '80% of the way' before RL fine-tunes the policy.
• Sample efficiency gap — Patel notes models may be a 'millionfold' behind humans in sample efficiency, comparing data seen from birth to adulthood versus a model's training corpus, as a potential hard limit on continual learning from deployment.
• Data vs architecture study — Millidge and Princeton student Jerry Han ran a grid pairing training recipes and datasets from 2019 to now and found data explains roughly a 12x compute-efficiency gain versus about 3.7x from architecture improvements at small scale.
• Talkie 1930 experiment — A model trained only on data up to 1930, fine-tuned on modern coding-agent data, reportedly outperformed Claude 3 Opus on SWE-bench, though a separate model trained only up to fifth-grade math failed to climb via RL to college-level math.
• Rapid-fire timelines — On a drop-in remote worker across white-collar tasks the panel says roughly 1-3 years; on 10x AI-researcher productivity uplift Millidge and O'Neill say about 2 years while Schulman says 5-10 years; on ASI dominating all computer-based cognitive work estimates cluster around 3-10 years, with Schulman calling automating AI research itself 'ASI-complete.'

In their words

But there were so many discrete discontinuities and innovations that had to happen to keep that scaling law going. The same thing has happened with LLMs.3:12

If not, we're probably going to hit this asymptotic curve.3:50

They're plausibly a millionfold behind humans in terms of how much data a human sees from birth to adulthood versus how much a model sees from cold start to finishing training.46:12

Even if the model can write way more code than a person, it doesn't make you 100X more productive.2:06

Disclosure · All three guests work at AI labs building the models discussed — Millidge is CTO of Zyphra, Schulman is chief scientist at Thinking Machines and a former OpenAI co-founder, and O'Neill is head of model training at Baseten — giving them direct commercial stakes in the trajectory they describe.

One thing to add — One thing to add — the panel's timeline estimates diverge by as much as 5x on the same underlying question of automating AI research, and that spread among insiders with direct access to frontier training internals is itself the most telling data point in the conversation. It suggests the uncertainty is not about facts withheld from outsiders but about a genuinely unresolved technical question.

One thing to try tonight
Read up on Beren Millidge's argument that RL's power comes mainly from mid-training warm starts rather than the RL step itself, then check it against John Schulman's "one bit per episode" framing to see which explanation better fits a model you use daily.