Lex Fridman PodcastLeslie Kaelbling: Reinforcement Learning, Planning, and Robotics | Lex Fridman Podcast #15
CHAPTERS
- 0:00 – 1:13
GEB, philosophy, and the early spark for AI
Leslie Kaelbling describes how reading Gödel, Escher, Bach in high school shaped her fascination with intelligence emerging from simple primitives. She and Lex connect that early interest to logic, reasoning, and the broader question of what kinds of programs can generate intelligent behavior.
- •Gödel, Escher, Bach as a formative influence
- •Intelligence from composing simple primitives into complex behavior
- •Early focus on reasoning/logic rather than robots
- •Framing AI as programmatic generation of intelligent behavior
- 1:13 – 3:05
From philosophy at Stanford to robotics at SRI
Kaelbling recounts studying philosophy at Stanford (before CS was a standard major there) and then transitioning into computer science. Her first job at SRI, working on a Shakey follow-on robot, pulled her into robotics through hands-on engineering needs.
- •Stanford philosophy background and its relevance to AI
- •Symbolic systems: logic, model theory, formal semantics
- •Joining SRI AI Lab and being tasked with making a robot work
- •Robotics interest driven by real systems and constraints
- 3:05 – 5:42
What philosophy contributes to AI (and what it doesn’t)
The discussion turns to whether AI researchers should also be philosophers. Kaelbling emphasizes the formal parts of philosophy (belief, knowledge, denotation) as directly relevant, while rejecting the idea that human-level robotics requires anything beyond technical advances.
- •Formal philosophy topics closely tied to AI representations
- •Materialist view: no fundamental barrier to humanlike behavior
- •“Zombie” question deemed less important than capability
- •Gap between robots and humans framed as technical, not philosophical
- 5:42 – 7:22
Shakey the robot: foundational ideas in planning and navigation
Kaelbling explains why Shakey remains iconic and urges people to read the Shakey Technical Report. She details Shakey’s capabilities—planning, replanning, perception, localization, and multiple abstraction levels—highlighting how many modern ideas were already present.
- •Shakey’s influence: A* search, symbolic planning, macro-operators
- •Robot capabilities: pushing objects, mapping, localization, replanning
- •Multi-level representations: low-level grids to symbolic rooms/connectivity
- •Perception-driven decision making (e.g., when to look/take pictures)
- 7:22 – 9:12
Flaky and “situated computation”: learning robotics by reinventing wheels
Moving from Shakey to Flaky, Kaelbling describes building a new robot at SRI essentially from scratch. She introduces situated automata as a design philosophy and argues that reinventing wheels can be a valuable learning process for developing real robotic systems.
- •Flaky as a fresh start without the original Shakey team
- •Situated computation: logic for designers’ analysis vs robot-internal theorem proving
- •Learning robotics fundamentals without prior control/sensing background
- •Wheel reinvention as a pathway to deeper understanding
- 9:12 – 11:43
AI’s oscillating history: cybernetics, expert systems, and shifting problems
Kaelbling sketches a cyclical history of AI where methods and even problem definitions go in and out of fashion. She contrasts early cybernetics/control with expert systems, and explains how communities often “shelve” hard problems or reframe them as malformed.
- •Fashion cycles in AI methods and research focus
- •Cybernetics/homeostasis as an early inspiration
- •Expert systems as superficial symbolic success that didn’t generalize
- •When approaches fail, the field often changes problems as well as methods
- 11:43 – 15:17
Why expert systems hit a wall—and what “symbolic” should really mean
Kaelbling argues expert systems failed largely because humans can’t truly articulate the knowledge they use to make decisions. She separates that critique from the usefulness of formal reasoning, pivoting toward the central role of abstraction rather than ideology about “symbolic vs neural.”
- •Key bottleneck: extracting real decision knowledge from humans
- •Post-hoc explanations vs true causal decision rules
- •Symbolic reasoning still useful, but terminology is often sloppy
- •Abstraction (spatial/temporal) is essential for tractable reasoning
- 15:17 – 18:04
Abstractions for planning: shrinking state, horizon, and complexity
The conversation dives deeper into abstraction as the mechanism that makes long-horizon planning possible. Kaelbling explains how agents reason at coarse levels (rooms, afternoons) and then switch to fine-grained continuous control when needed, advocating a pragmatic mix of representations.
- •State/horizon reduction as the core motivation for abstraction
- •Spatial abstraction (room vs pose) and temporal abstraction (afternoon vs exact time)
- •Switching representations depending on the moment and task
- •Rejecting dogma: use whatever computational tools work
- 18:04 – 21:45
MDPs and POMDPs: modeling stance, uncertainty, and belief updates
Kaelbling frames MDPs and POMDPs as modeling choices (a “stance”), not literal descriptions of reality. She defines MDPs via full state observability and Markov sufficiency, then explains POMDPs as reasoning from histories of actions/observations to infer hidden state.
- •Modeling as a stance: the world isn’t an MDP, but can be treated as one
- •MDP definition: fully observed state with Markov property and stochastic dynamics
- •POMDP definition: hidden state + observations; infer state from history
- •Decision-making under uncertainty via belief updating
- 21:45 – 23:22
Planning under uncertainty is intractable—so approximate intelligently
Kaelbling notes that optimal POMDP planning can be undecidable/intractable, but argues that doesn’t excuse ignoring it. Instead, she advocates using the formalism for clarity and then applying layered approximations in modeling and computation to reach workable solutions.
- •Optimal POMDP planning can be undecidable depending on formulation
- •“Intractable” is not a reason to avoid the real problem
- •Use the formal model to guide approximation choices
- •Approximate both the model and the solution algorithm for practicality
- 23:22 – 26:30
From engineering to science: bounded optimality, theory gaps, and what guarantees mean
The discussion shifts to methodology: modern AI’s engineering success is outpacing its scientific understanding. Kaelbling argues the field lacks strong approximate-solution concepts for very hard problems, and calls for principles that predict when/why systems work (like bridge engineering).
- •Current imbalance: empirical progress without deep theory
- •Need better approximate solution concepts and performance guarantees
- •“Very bad job at very big problems” as a research posture
- •Turning hacks into science: principles for predicting success/failure
- 26:30 – 29:20
Belief space control: acting to change what you know, not just the world
Kaelbling explains belief space as treating the agent’s belief (a distribution over states) as the object being controlled. This enables deliberate information-gathering actions and risk-aware behavior based on uncertainty, illustrated with driving and navigation examples.
- •Belief state as a probability distribution over world states
- •Control problem becomes: choose actions to shape beliefs and outcomes
- •Information-gathering decisions (e.g., look over shoulder while driving)
- •Uncertainty-aware planning (don’t go through a door if you’re too unsure)
- 29:20 – 35:01
Hierarchical planning in the real world: airports, feasibility leaps, and generalization
Returning to long-horizon behavior, Kaelbling explains temporal hierarchy: high-level plans depend on learned expectations that subgoals are feasible without detailing every step. She uses navigating airports to motivate generalization—predicting effort/time in unfamiliar environments.
- •Temporal hierarchy: decomposing long tasks into abstract segments
- •Planning at high level requires belief in feasibility without full detail
- •Airport navigation as an example of unavoidable underspecification
- •Research focus: learning abstract models to predict subgoal difficulty/time
- 35:01 – 41:10
Model-based vs model-free, why perception is harder than planning, and building useful structure
Kaelbling argues intelligence won’t come from one algorithm but from multiple representations and reasoning styles, with model-based/model-free tradeoffs as time–space computation choices. She states perception is harder than planning because the key challenge is representational—what perception should output and what inductive biases (like convolution, objects, relational structure) should be built in.
- •No “one true algorithm”: integrate multiple reasoning/learning styles
- •Model-free policy vs value vs model+planner as computation trade-offs
- •Perception is harder: representation and interface to the rest of the system are unclear
- •Need strong inductive biases beyond convolution (objects, relational/graph structure)
- 41:10 – 1:01:23
Human-level robotics, benchmarks, and AI futures: value alignment and research incentives
In the final stretch, Kaelbling downplays consciousness as a prerequisite while acknowledging practical “self-monitoring” is necessary. She critiques benchmark/competition culture as sometimes hack-driven, discusses founding JMLR and rethinking peer review, and closes with thoughts on AI cycles, overselling, and near-term importance of objective functions and value alignment.
- •Consciousness not central; practical self-monitoring/meta-control matters
- •Skepticism about benchmarks/competitions, but acknowledges they can spark ideas
- •JMLR origin: open-access alternative to expensive publisher model; curation vs “publishing”
- •AI boom-bust cycles likely; alignment and objective-function engineering as key near-term issues
