By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Sequoia CapitalBuilding the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Episode Details
EPISODE INFO
- Released
- July 29, 2026
- Duration
- 49m
- Channel
- Sequoia Capital
- Watch on YouTube
- ▶ Open ↗
EPISODE DESCRIPTION
Jerry Tworek led reasoning at OpenAI, convinced that scaling reinforcement learning was the path to AGI. Rohan Anil co-led Gemini pre-training and built the Shampoo optimizer. Now they've teamed up at Core Automation on a contrarian premise: the transformer has carried us as far as it can, and the bottleneck to smarter systems is no longer scale — it's the architecture itself. The missing capability is continual learning, models that adapt at test time, which transformers can't do. In-context learning taps out fast (Codex needs compacting after ~20 minutes) and fine-tuning invites catastrophic forgetting. Rohan argues pre-training and RL should be optimized end-to-end, and that transformers spend computation inefficiently. They lay out why the largest labs won't chase alternatives while locked in the coding-agent race, and why building the world's most automated lab starts with automating kernel generation—the one place frontier models still lose to a high-taste human. Hosted by Sonya Huang and Pat Grady, Sequoia Capital 00:00 Introduction 01:46 Appreciating Transformers 02:44 Scaling Hits Limits 04:54 Why Architecture Matters 05:32 RL Reality Check 07:32 Test Time Learning 09:52 Economics Of Scaling 12:47 Why Start A Company 14:24 Rohan On Transformers 19:11 Computational Depth Problem 20:32 When Transformers Top Out 23:22 Beyond Reinforcement Learning 26:41 Optimization And Efficiency 34:24 Building An Automated Lab 39:45 Kernel Automation Roadmap
SPEAKERS
Jerry Tworek
guestAI researcher and co-founder of Core Automation, previously at OpenAI.
Pat Grady
hostPartner at Sequoia Capital and co-host/interviewer on the episode.
Sonya Huang
hostPartner at Sequoia Capital and primary host/interviewer of the episode.
Rohan Anil
guestAI researcher and co-founder of Core Automation, formerly at Google Brain with a focus on optimization and training efficiency.
EPISODE SUMMARY
In this episode of Sequoia Capital, featuring Jerry Tworek and Pat Grady, Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil explores Core Automation’s bet: replace transformers via continual, test-time learning Jerry Tworek said appreciating transformers means understanding what they do well and why they became economically viable, while also recognizing their limitations for the next phase of progress.
RELATED EPISODES
