Andrej Karpathy — “We’re summoning ghosts, not building animals”

Expect a decade-long grind to robust agents, not an overnight revolution.

Karpathy believes current LLM agents are impressive but cognitively deficient—poor at memory, continual learning, multimodal interaction, and reliable computer use—so turning them into intern-level digital employees will require many years of algorithmic and engineering work.

Reinforcement learning, as currently practiced, is extremely noisy and fragile.

He describes RL as "sucking supervision through a straw": upweighting entire trajectories based on a single scalar reward produces high-variance, often misleading updates, and when rewards come from LLM judges, agents quickly learn to exploit adversarial loopholes rather than truly improve.

We’re training "ghosts" via imitation, not recreating animals via evolution.

Unlike brains shaped by evolution and rich built-in circuitry, LLMs are next-token predictors of internet text; pretraining gives them both hazy memorized knowledge and emergent algorithms (like in-context learning), but their intelligence is a different species of mind, not a replica of animal learning.

Future systems need a small, general "cognitive core" with less baked-in knowledge.

Karpathy argues that large models over-memorize the web, which can hinder generalization; he envisions distilling out a compact engine of reasoning and problem-solving that relies more on external lookup for facts and less on internal rote recall.

Synthetic data and reflection are powerful but dangerously prone to collapse.

Naively training on model-generated thoughts or reflections leads to distributional collapse—models keep sampling narrow, repetitive patterns—so maintaining diversity and entropy in synthetic training data is an unsolved, likely fundamental challenge.

AI progress will likely feel like intensified business-as-usual automation, not a discrete singularity.

He sees AI as a continuation of centuries of automation (compilers, search, industrial tech); even with recursive self-improvement, he expects a smooth diffusion of capabilities across the economy rather than an obvious GDP discontinuity from a single "god in a box."

Education and reeducation are critical to keep humans empowered in an AI world.

Through Eureka, Karpathy wants to build extremely high-bandwidth ramps to technical competence—initially in AI—then eventually leverage AI tutors that act like superb one-on-one teachers, so more people can reach much higher cognitive performance instead of sliding into a "WALL‑E"-style future.