Lex Fridman PodcastTuomas Sandholm: Poker and Game Theory | Lex Fridman Podcast #12
CHAPTERS
- 0:00 – 4:10
Heads-up No-Limit Texas Hold’em as an AI benchmark (rules + why it’s hard)
Lex and Tuomas lay out the basic structure of heads-up no-limit Texas Hold’em and why it became a flagship benchmark for imperfect-information game solving. They highlight the sequential revelation of public cards, private hole cards, and why hidden information changes everything compared to chess or Go.
- •Heads-up = two-player, highly competitive format; mostly played online for high stakes
- •Imperfect information comes primarily from private hole cards
- •Four betting rounds with staged information revelation (preflop, flop, turn, river)
- •Why imperfect-information games are fundamentally harder than perfect-information board games
- 4:10 – 7:08
The Libratus vs. top pros match: setup, incentives, and interface
Tuomas recounts the 20-day Rivers Casino match where Libratus played four elite specialists over 120,000 hands to reach statistical significance. They discuss how the event was structured, how pros were paid, and the practical UI details that ensured fairness.
- •Brought in four of the top ten heads-up no-limit specialists to Pittsburgh
- •20 days, target of ~120,000 hands for statistical significance
- •$200K incentive pool paid out by performance against the AI
- •Online-style interface with full betting history to remove “memory advantage” concerns
- •Regulatory constraints prevented playing for real money
- 7:08 – 10:24
Confidence, betting markets, and the myth of “poker tells”
They explore Tuomas’s uncertainty going into the match, including prior failure with Cloudico, and how betting markets underestimated the AI. The conversation then pivots to why people romanticize poker as uniquely human due to tells and psychology, and why that matters less at elite levels.
- •Earlier system (Cloudico) lost; Libratus was stronger but outcome uncertain
- •Tuomas estimated a ~50/50 chance; betting markets made AI a 4–5:1 underdog
- •Even after early wins, markets stayed skeptical—humans over-trust human skill
- •Hollywood “tells” vs reality: top players hide tells effectively
- •At high levels, betting patterns and strategy dominate over physical reads
- 10:24 – 12:59
Scaling to 10^161: information vs action abstraction (and abstraction pitfalls)
Tuomas explains why solving poker requires abstraction and why abstraction in games is trickier than in single-agent settings. They distinguish information abstraction (cards) from action abstraction (bet sizes) and describe how Libratus approached each—while warning about counterintuitive pathologies.
- •Full game is ~10^161 decision points—direct equilibrium solving is infeasible
- •Game abstractions can backfire: finer abstraction can yield worse real-game performance
- •Information abstraction (cards): automated, “potential-aware” clustering over future strength
- •Action abstraction (bets): based on observed play; limited use of provably convergent bet sizing early
- •Clear split: chance abstraction vs player-action abstraction
- 12:59 – 14:27
Luck, variance, and why you need 100,000+ hands
They discuss the balance between card strength and betting skill, emphasizing poker’s high variance—especially in no-limit all-in situations. This variance explains the need for extremely long matches to confidently determine who is better.
- •Romantic claim ‘actions matter more than cards’ has limits—cards still matter
- •No-limit creates massive variance due to large all-in swings
- •Even thousands of hands can be misleading; variance remains high
- •Statistical significance requires ~100,000+ hands in this domain
- 14:27 – 18:57
Learning vs game-theoretic solving: why imperfect information complicates value functions
Lex asks why Libratus didn’t use deep learning and how learning could fit into poker. Tuomas explains that in imperfect-information games, the value depends on beliefs (not just states), making AlphaGo-style evaluation functions insufficient without modeling information sets and opponent beliefs.
- •Libratus originally used no deep learning; later work explores deep learning components
- •In imperfect information, value depends on both players’ belief distributions
- •A bad hand can be valuable if the opponent believes it’s strong (and vice versa)
- •Depth-limited lookahead is hard because leaf values must account for strategic beliefs
- •Comparing approaches: Libratus endgame solving vs DeepStack’s learned belief-aware values
- 18:57 – 22:43
Beliefs, Bayes, and Nash equilibrium: ‘beliefs are output, not input’
They unpack how belief distributions arise naturally from rules (uniform dealing + card removal) and are refined via Bayes’ rule under equilibrium strategies. Tuomas uses rock–paper–scissors to illustrate how a strategy implies beliefs, reinforcing that the approach doesn’t require player-specific data.
- •Initial beliefs come from game rules; card removal is crucial (single deck constraints)
- •Nash equilibrium defines rational strategies and induces beliefs via Bayes’ rule
- •Rock–paper–scissors example shows strategy → belief distribution at an information set
- •Game theory is largely ‘data-free’ and not based on opponent history
- •Distinction between modeling rationality vs modeling a specific opponent
- 22:43 – 25:13
Opponent exploitation: hybridizing equilibrium play with data (and why Libratus avoided it)
Tuomas describes work that blends safe equilibrium strategies with selective exploitation based on observed weaknesses. He also explains why Libratus didn’t use exploitation against top pros: exploiting can open you up to counter-exploitation, and elite opponents are best at punishing that.
- •Equilibrium strategies are unexploitable but not maximally profitable vs weak players
- •Hybrid approach: start with equilibrium; shift locally toward exploitation with evidence
- •Tradeoff: more exploitation usually increases vulnerability
- •Libratus kept exploitation off due to pros’ counter-exploitation expertise
- •Applicability beyond poker: negotiations, strategic interactions, and other domains
- 25:13 – 30:22
Taxonomy of games and why multiplayer/general-sum is a major leap
The discussion broadens into a map of game classes: repeated matrix games, stochastic games, and extensive-form games. Tuomas explains why two-player zero-sum games are comparatively clean, while general-sum or multi-player games introduce equilibrium selection, coordination problems, and collusion.
- •Repeated games are rare as exact models of reality; real settings change over time
- •Stochastic games sit between matrix and extensive-form: actions determine distributions over next states
- •Extensive-form games represent sequential decision trees with information sets
- •Two-player zero-sum: any equilibrium works well against any equilibrium (conceptual simplicity)
- •General-sum/multiplayer: multiple equilibria, selection issues, collusion incentives
- 30:22 – 32:45
Collusion and coordination: why cooperative poker variants explode in difficulty
Lex asks why allowing collaboration would make poker dramatically harder, referencing Noam Brown’s comments. Tuomas connects this to coalitional game theory and gives bridge as an example of team payoffs with restricted communication, highlighting the modeling and computational challenges.
- •Collusion changes the problem class and often breaks standard representations
- •Bridge example: shared payoffs, no direct communication, only pre-coordination and public signals
- •Auctions/negotiations/diplomacy/poker can benefit from collusion even if not built into rules
- •New representations may be needed to compute effectively under coalition structure
- •Collusion adds uncertainty about who coordinates with whom and how
- 32:45 – 37:55
From poker to practice: startups, negotiation, and autonomous-vehicle coordination
Tuomas outlines how game-solving ideas translate into business, finance, security, and military contexts through his startups. They also discuss a concrete autonomous-vehicle angle: pre-negotiating merging behaviors across fleets via automated, combinatorial negotiation rather than ad hoc human-style interaction.
- •Strategic Machine: business, sports, gaming, finance, electricity markets
- •Strategy Robot: military, security, cyber, and intelligence applications
- •Automated negotiation for fleets could enable faster, safer merging via pre-agreed protocols
- •Real-world systems leave large strategic latitude even when rules are fixed
- •Key question: model as perfect vs imperfect information depending on what’s observable
- 37:55 – 43:16
Performance-oriented research: why scaling experiments matter (and the poker backlash)
Tuomas argues that theory and small-scale experiments can mislead; large systems and real evaluation reveal what truly works. He recounts the intense negative reaction from parts of the poker community, then reframes AI as enriching the game—humans adopt the ‘Martian’ strategies AIs discover.
- •Many techniques look good small-scale but fail at full size; theory can be non-predictive
- •CFR-based methods often win in practice despite theory favoring others
- •Backlash: fear superhuman poker bots would undermine trust in the game
- •Analogy to chess: humans keep playing even after engines surpass them
- •AI strategies changed human poker, making it richer and more nuanced
- 43:16 – 48:31
Automated mechanism design: impossibility results and ‘islands of possibility’
They shift to mechanism design—designing the rules to achieve desired outcomes—and Tuomas emphasizes both promise and limits. He explains that impossibility theorems rule out achieving certain properties across broad classes, but automated design can still find workable mechanisms for specific settings, including within otherwise ‘impossible’ classes.
- •Automated mechanism design is powerful but not a panacea
- •Impossibility results are formal proofs, not limits of creativity
- •Designing for specific instances can bypass class-level impossibilities
- •Example: navigating around the Myerson–Satterthwaite impossibility in many settings
- •Real deployments often use theory as inspiration rather than strictly truthful mechanisms
- 48:31 – 54:10
What’s next for AI/game solving: benchmarks, real-world strategy, and interpretability
Lex asks about the next milestone after poker; Tuomas mentions candidates like StarCraft, DOTA 2, Diplomacy, and Hanabi but notes the community lacks a single agreed benchmark. He then argues the bigger breakthrough is widespread real-world strategic planning, and discusses the tension between provable guarantees and human interpretability.
- •After heads-up no-limit, no universally accepted next benchmark
- •Candidate domains: StarCraft, DOTA 2, Diplomacy, Hanabi
- •Goal shifts from recreational games to real-world strategic decision-making
- •Game-theoretic solutions offer guarantees but can still be hard to understand
- •Belief: much of business/military strategy could eventually be computed strategically
- 54:10 – 1:06:17
AI risk, societal impact, and the threats Tuomas worries about most
They close on AI safety and societal consequences: Tuomas is optimistic, citing kidney exchange and supply-chain efficiency as tangible benefits, and sees “value misalignment” as mostly theoretical in his applied experience. He identifies climate change and nuclear war as primary existential risks, and explains why nuclear stability may be more fragile today despite MAD logic.
- •Optimism grounded in applications: kidney exchange saves lives; logistics auctions improved efficiency
- •Value misalignment framed as an objective-function design issue; rarely seen as a real deployed failure
- •AI could also improve safety (security, planning), often overlooked in public debates
- •Top global risks in his view: climate change and nuclear war
- •MAD is a coarse model; proliferation, smaller actors, and lower thresholds increase risk
