No Priors Ep. 80 | With Andrej Karpathy from OpenAI and Tesla

Karpathy notes Waymo drove him flawlessly a decade ago, yet it took 10 years to become a commercial product—highlighting that moving from a polished demo to robust, global deployment demands years of engineering, edge-case handling, and regulatory work.

He argues Tesla has a “software problem” (easier to fix) while Waymo has a “hardware problem” (scaling expensive sensor rigs), and that Tesla’s massive installed base plus vision-only deployment—augmented by rich sensors at training time—is a powerful arbitrage.

Unlike earlier architectures, transformers scale cleanly and act like a general-purpose differentiable computer; most innovation now happens in dataset curation and objective design rather than in model architecture.

He warns that naive synthetic data pipelines produce narrow, repetitive distributions (e.g., LLMs repeating the same few jokes), so future training must deliberately inject diversity—via personas, varied prompts, and other techniques—to maintain richness.

Tesla’s robot reused car hardware, software, and data infrastructure so seamlessly that early Optimus “thought it was a car”; Karpathy believes a single humanoid platform maximizes reuse, teleoperation ease, and cross-task transfer learning.