Lex Fridman PodcastTomaso Poggio: Brains, Minds, and Machines | Lex Fridman Podcast #13
CHAPTERS
- 0:00 – 3:39
Einstein, thought experiments, and the power of nonconformity
Lex introduces Tomaso Poggio and opens with Poggio’s childhood admiration for Einstein and relativity. Poggio frames Einstein’s genius as the ability to reach deep truths through thought experiments and notes how being an outsider/nonconformist can enable scientific breakthroughs.
- •Einstein’s relativity as a triumph of thought experiments (gedanken experiments)
- •Imagination/visualization as a core ingredient of discovery
- •Einstein as an “anti-top-student”: unconventional thinking over credentials
- •Nonconformity as an advantage in science (and a humorous stock-market analogy)
- 3:39 – 6:14
Time travel skepticism and the broader dream of building intelligence
The conversation shifts from relativity’s mysteries to time travel and what physics might allow. Poggio is skeptical about traveling back in time, but reaffirms the ambition of machines that can think as well as humans—or help humans think better.
- •Time travel: maybe forward via relativity/cryonics-like ideas, unlikely backward
- •Some problems may be physically impossible; intelligence may not be
- •Machines as cognitive amplifiers in the near/mid term
- •“Intelligence” is a tricky concept that needs careful definition
- 6:14 – 8:46
Why intelligence is the biggest scientific problem
Poggio explains why understanding intelligence captivates him even more than cosmology or the origin of life. His early motivation was that solving intelligence could produce tools (or minds) capable of solving many other hard scientific problems.
- •Intelligence as a meta-problem: understanding the tool used to do science
- •Early “teenage” motivation: build an Einstein-times-10 problem solver
- •Questions about the brain: origins, limits, and how to improve it
- •Curiosity about what humans are and how cognition emerges
- 8:46 – 13:07
Can we build strong AI without understanding the brain? Lessons from flight
Lex asks whether AGI can be engineered without deep biological understanding. Poggio compares this to building airplanes without fully copying birds, but argues that recent AI progress has been strongly driven by neuroscience inspiration, making it an open ‘educated bet.’
- •Analogy: flight succeeded without fully replicating bird biology
- •AGI timeline and capability are uncertain; routes to it are debated
- •Neuroscience’s role in recent AI breakthroughs (RL, deep learning)
- •Prediction: neuroscience will likely continue to inspire some key advances
- 13:07 – 17:16
Biological vs artificial neural nets: what’s missing today (labels, data, learning)
Poggio contrasts modern deep nets with biological learning and argues that today’s major weakness is reliance on massive labeled datasets. He highlights how children learn from very few labeled examples, motivating the “N→1” challenge.
- •Deep nets are simplified, but closer to brains than symbolic logic systems
- •Main gap: supervised deep learning needs enormous labeled datasets
- •Children learn concepts from sparse labels; ImageNet-style labeling is unnatural
- •Framing: AI today is “N→∞” labeled data, biology is closer to “N→1”
- 17:16 – 22:36
Nature vs nurture in learning: evolution, priors, and face-recognition plasticity
The discussion turns to how much learning is hardwired versus learned, using genetics and evolution as context. Poggio describes experiments suggesting face-selective brain areas aren’t prewired as ‘face templates’ but emerge from early-life imprinting in a plastic region.
- •Nature vs nurture: evolution provides priors plus learning machinery
- •DNA gene count comparison (humans vs fruit fly) as an intuition pump
- •Face-area development: monkey experiments with early face deprivation
- •Hypothesis: genes specify easy-to-imprint plasticity, not detailed face circuitry
- •Evidence of alternative tuning (e.g., glove/hand sensitivity) with altered exposure
- 22:36 – 27:55
Is the brain modular or uniform? Cortex as shared “hardware” across functions
Poggio rejects the old ‘equipotential brain’ idea and affirms specialized modules, while also emphasizing cortical uniformity across modalities. The cortex appears to reuse similar circuitry for very different tasks (vision, language, motor), raising deep questions about shared computational principles.
- •Historical context: Lashley’s equipotentiality vs modern modular evidence
- •Lesions, war injuries, and fMRI reveal functional specialization
- •Cerebellum/hippocampus differ anatomically; cortex looks more uniform
- •Open puzzle: how the same cortical ‘hardware’ supports diverse functions
- •Implication for AI: seek architectures that generalize across domains
- 27:55 – 32:51
Vision as a gateway to intelligence—and why understanding brains needs many levels
Lex asks how the visual cortex builds understanding from sensory input, and Poggio stresses both how much we know and how many basics remain mysterious (e.g., sleep). They discuss levels of abstraction using a computer analogy, arguing brain ‘hardware’ and ‘software’ are more intertwined than in engineered computers.
- •We know many details about vision, but big basic questions remain (e.g., sleep)
- •Different levels of understanding are all valuable (transistors vs PowerPoint)
- •In computers, abstraction layers are intentionally separable; in brains, less so
- •Brain science requires collaboration across theory, experiments, and computation
- •The brain as a difficult but potentially decomposable ‘hierarchical mess’
- 32:51 – 35:47
Compositionality: when deep networks beat shallow ones
Poggio explains the theoretical idea that deep networks excel when the target function is compositional—built from local computations composed hierarchically. Vision and language naturally fit this structure, and this provides a lens on why depth can defeat the curse of dimensionality for certain problem classes.
- •Deep vs shallow: depth helps when functions are compositions of subfunctions
- •Local-to-global computation: pixels→parts→objects analogy
- •Language analogy: letters→syllables→words→sentences
- •Compositional structure as a sufficient condition for exponential efficiency gains
- •A theory-driven justification for hierarchical architectures (e.g., convnets)
- 35:47 – 39:17
Why compositionality exists: physics, brain wiring limits, and evolution
Poggio and Lex debate whether compositional structure comes from the physical world (local interactions) or from constraints of brain wiring that bias what problems humans can solve. Poggio suggests biology’s short-range connectivity and limited long-range wiring may have shaped cognition toward deep, local architectures.
- •Max Tegmark’s view: compositionality reflects locality in physics
- •Alternative view: brains are deep/local, so we focus on problems they can solve
- •Biological constraints: long-range connections are expensive/limited
- •Deep convolutional-like organization as a byproduct of wiring constraints
- •Evolution: ‘good enough’ compositional problems supported survival and skill
- 39:17 – 44:47
Stochastic gradient descent: why it works, and why biology might do something else
Lex presses on why SGD is so effective despite seeming biologically implausible. Poggio argues that over-parameterization creates an enormous number of global minima, making optimization easier, while noting that why solutions generalize well is a separate and deeper question.
- •Cortex architecture aligns with deep nets; SGD likely not biologically literal
- •Open problem: find alternative learning/optimization mechanisms
- •Modern shift: models often have far more parameters than data
- •Over-parameterization implies huge numbers of (near-)global minima
- •Optimization may be easy; selecting minima that generalize is the hard part
- 44:47 – 47:50
Universal approximation, the curse of dimensionality, and how depth can avoid it
Poggio downplays universal approximation as unsurprising (akin to Weierstrass), emphasizing that efficiency—not mere approximability—is what matters. He explains how shallow approximators scale disastrously with dimension, and claims deep hierarchical models can avoid this curse when the target function is compositional.
- •Universal approximation is like Weierstrass: existence is not the point
- •Key question: how many units/terms are required for a desired error?
- •Curse of dimensionality: requirements blow up exponentially with input dimension
- •Deep/local hierarchies can avoid the curse for compositional functions
- •Theory connects architectural bias with feasible learning in high dimensions
- 47:50 – 51:13
Unsupervised learning and GANs: impressive outputs vs the real ‘N→1’ challenge
Lex asks about GANs and unsupervised learning as routes to reducing labeling needs. Poggio is cautious, arguing GANs are valuable for generating realistic images and density estimation but may not solve the fundamental problem of learning with few labels.
- •GANs as a novel density-estimation/generative approach
- •Skepticism: photorealistic generation doesn’t imply better sample efficiency
- •GANs likely useful for graphics and related applications
- •Poggio’s earlier work on generating images from pose/expression inversion
- •“No free lunch” intuition: unsupervised methods can’t magically create supervision
- 51:13 – 55:05
How babies learn: bootstrapping weak priors using motion and segmentation
Poggio proposes a developmental story: evolution provides weak priors (like motion sensitivity), which enable early segmentation of objects from background, accelerating learning. This bootstrapping view suggests how AI might reduce labeling demands by exploiting self-supervised structure in sensory data.
- •Learning as bootstrapping from weak, evolution-shaped priors
- •Motion detection as a highly conserved early capability across species
- •Motion enables automatic segmentation (object vs background)
- •Early segmentation makes object learning far easier than static scenes
- •Implication: leverage structure in data (e.g., temporal continuity) to learn faster
- 55:05 – 1:02:41
Limits of today’s AI: scene understanding, existential risk, and AGI timelines
Poggio distinguishes today’s successes in low-level perception from true scene/language understanding, which he считает far away. He supports early safety thinking but rejects comparisons that AI is more dangerous than nuclear weapons, and he offers cautious (century-plus) AGI timeline intuitions and reflections on explainability.
- •Object recognition vs ‘understanding’: current systems remain far from the latter
- •We’re in a ‘golden age’ of applications, but deep understanding is missing
- •AI risk: worry early, but don’t misprioritize vs nuclear weapons
- •AGI timelines: hard to predict; Poggio leans toward very long horizons
- •Understanding vs building: we may build learning systems without predicting discoveries
- 1:02:41 – 1:20:20
Ethics, consciousness, mortality—and closing advice on science and mentoring
The conversation moves into ethics and consciousness: Poggio argues ethics is likely learnable and points to brain areas tied to moral judgment, while noting consciousness is hard to define and may or may not be required for intelligence. The closing section covers MIT ‘moonshots’ in visual intelligence, a VR self-location experiment, Poggio’s philosophy of mentorship, and reflections on intelligence and happiness inspired by Flowers for Algernon.
- •Neuroscience of ethics: correlates, stimulation effects, and why it matters for AI
- •Consciousness debates (biological vs silicon; Tononi/Koch perspectives)
- •Mortality as a possible motivator but not obviously required for consciousness
- •MIT/CBMM focus: visual intelligence and fast perception of environments
- •VR experiment: self-location can ‘move’ with sensory remapping
- •Success in science: curiosity, fun, and collaborative environments; mentoring style
- •Final reflections: intelligence as gift/curse and its uncertain link to happiness
