Noam Brown: AI vs Humans in Poker and Games of Strategic Negotiation | Lex Fridman Podcast #344

Noam Brown 0:00 A lot of people were saying, like, "Oh, this whole idea of game theory, it's just nonsense and if you really want to make money, you gotta, like, look into the other person's eyes and read their soul and figure out what cards they have." But what happened was w- where we played our bot against four top heads of no-limit hold 'em poker players, and the bot wasn't trying to adapt to them. It wasn't trying to exploit them. It wasn't trying to do these mind games. It was just trying to approximate the Nash equilibrium, and it crushed them.

Lex Fridman 0:28 The following is a conversation with Noam Brown, research scientist at FAIR, Facebook AI Research Group at Meta AI. He co-created the first AI system that achieved superhuman level of performance in no-limit Texas hold 'em both heads up and multiplayer. And now, recently, he co-created an AI system that can strategically out-negotiate humans using natural language in a popular board game called Diplomacy, which is a war game that emphasizes negotiation. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description, and now, dear friends, here's Noam Brown.

Lex Fridman 1:09 You've been a lead on three amazing AI projects. So we got Libratus that solved, or at least achieved human-level performance on, uh, no-limit Texas hold 'em poker with two players, heads up. You got Pluribus. That solved no-limit Texas hold 'em poker with six players, and just now you have Cicero. These are all names of systems that solved or achieved human-level performance on the game of Diplomacy, which, uh, for people who don't know, is a popular strategy board game. It was loved by JFK, John F. Kennedy and Henry Kissinger, and many other big famous people in the decades since. So let's talk about poker and Diplomacy today. First poker, what is the game of no-limit Texas hold 'em and how is it different from chess?

Noam Brown 2:00 Well no-limit Texas hold 'em poker is the most popular variant of poker in the world. So, you know, you go to a casino, you play, sit down at the poker table. The game that you're playing is no-limit Texas hold 'em. If you watch movies about poker like Casino Royale or Rounders, the game that they're playing is no-limit Texas hold 'em poker. Now it's very different from limit hold 'em in that you can bet any amount of chips that you want, and so the stakes escalate really quickly. You start out with, like, $1 or $2 in the pot and then by the end of the hand, you've got like $1000 in there maybe.

Lex Fridman 2:32 So the option to increase the number very aggressively and very quickly is always there?

Noam Brown 2:36 Right. The no-limit aspect is there's no limit to how much you can bet. You know, you, in limit hold 'em, there's like $2 in the pot. You- you can only bet, like, $2. But if you've got $10,000 in front of you, you're always welcome to put $10,000 into the pot.

Lex Fridman 2:50 So I got a chance to hang out with, uh, Phil Hellmuth who plays all these different variants of poker, and correct me if I'm wrong, but it seems like no limit rewards crazy versus the other ones rewards more kind of calculated strategy. Or- or no, because you're sort of looking from an- from an analytic perspective, is- is strategy also rewarded in no-limit Texas hold 'em?

Noam Brown 3:15 I think both variants reward strategy, but I think what's different about no-limit hold 'em is i- it's- it's much easier to get jumpy, you know. You go in there thinking you're gonna lose, you know, you're gonna play for, like, $100 or something and suddenly there's like, you know, $1000 in the pot. A lot of people can't handle that.

Lex Fridman 3:32 Can you define jumpy?

Noam Brown 3:33 When you're playing poker, you always want to choose the action that's going to maximize your expected value. It's kind of like- kind of like with investing, right? Like, if you're ever in a situation where you're the amount of money that's at stake is, um, is going to have a material impact on your life, then you're gonna play in a more risk averse style. You know, if somebody makes a huge bet, you're gonna... If you're playing no-limit hold 'em and somebody makes a huge bet, there might come a point where you're like, "This is too much money for me to handle. Like, I can't risk this amount." Uh, and that's what throws a lot of people off. So that's the big difference, I think, between no limit and limit.

Lex Fridman 4:10 Wha- what about on the action side when you're actually making that big bet? That's what I mean by crazy. I was- I was trying to refer to the technical, the (laughs) the technical term of crazy, meaning use the big jump in the bet to completely throw off the other person in terms of, um, their ability to reason optimally.

Noam Brown 4:30 I think that's right. I think one of the key strategies in poker is to put the other person into an uncomfortable position, and if you're doing that, then you're- you're playing poker well. And there's a lot of opportunities to do that in no-limit hold 'em. You know, you can have, like, $50 in there. You throw in a $1000 bet and, um, you know, that's sometimes, if you do it right, it puts the other person in a really tough spot. Now, it's also possible that you make huge mistakes that way, and so it's really easy to lose a lot of money in no-limit hold 'em if you don't know what you're doing. Um, but there- there's a lot of upside potential

Noam Brown 5:02 too.

Lex Fridman 5:02 So when you build systems, AI systems that play these games, we'll talk about poker, we'll talk about Diplomacy, are you, um, are you drawn in in part by the beauty of the game itself, AI aside? Or is it, to you, primarily a fascinating problem set for the AI to solve?

Noam Brown 5:20 I'm drawn in by the beauty of the game. Uh, when I... I started playing poker when I was in high school, and the idea to me that there is a correct, an objectively correct way of playing poker, and if you could figure out what that is then you're, you know, you're making unlimited money basically, that's like a really fascinating concept to me. Um, and so I- I was fascinated by the strategy of poker even when I was, like, 16 years old. It wasn't until, like, much later that I actually worked on poker AIs.

Lex Fridman 5:49 So there was a sense that you can solve poker, like, uh, in the way you can solve chess, for example, or checkers? I believe checkers got solved, right? I think so.

Noam Brown 5:57 Yeah, checkers- checkers is completely solved.

Lex Fridman 5:59 There's an op- optimal strategy for checkers.

Noam Brown 6:00 Optimal strategy. It's impossible to beat the AI.

Lex Fridman 6:01 Yeah, and so in that same way you could technically solve chess.

Noam Brown 6:05 You could solve chess, you could solve poker.

Lex Fridman 6:07 You could solve poker?

Noam Brown 6:08 So, this is... This gets into the concept of a Nash equilibrium.

Lex Fridman 6:12 Okay.

Noam Brown 6:12 So-

Lex Fridman 6:12 What is a Nash equilibrium?

Noam Brown 6:14 Okay. (laughs) So, in any finite two-player zero-sum game, there is an optimal strategy that, if you play it, you are guaranteed to not lose in expectation, no matter what your opponent does. And this is kind of a radical concept to a lot of people, um, but it's true in chess, it's true in poker, it's true in any finite two-player zero-sum game. And to give some intuition for this, you can think of rock-paper-scissors. In rock-paper-scissors, if you randomly choose between throwing rock, paper, and scissors with equal probability, then no matter what your opponent does, you are not going to lose in expectation. You are not going to lose in expectation in the long run. Now, the same is true for poker. There exists some strategy, some really complicated strategy, that if you play that, you are guaranteed to not lose money in the long run. And I should say, this is for two-player poker. Six-player poker is a different story.

Lex Fridman 7:03 Yeah, it's a beautiful, giant mess. When you say "in expectation," you're guaranteed not to lose in expectation. What does "in expectation" mean?

Noam Brown 7:12 Poker is a very high variance game, so you're gonna have hands where you win, you're gonna have hands that you lose. Even if you're playing the perfect strategy, you can't guarantee that you're gonna win every single hand. But if you play for long enough, then you are guaranteed to at least break even and, and in practice, probably win.

Lex Fridman 7:27 So, that's in expectation, the size of your stack, generally speaking. Now, that doesn't include anything about the fact that you can go broke. It doesn't include any of those kinds of normal real-world limitations. You're talking in a, in the theoretical world. Uh, what about this, the zero-sum aspect? How big of a constraint is that, and how big of a constraint is finite?

Noam Brown 7:48 So, finite's n- not a, a huge constraint. So, I mean, most games that you play are finite in size. Um, it's also true actually, that there exists this like, perfect strategy in many infinite games as well. Technically, the game has to be compact. Um, there are like, some edge cases where you don't have a Nash equilibrium in a two-player zero-sum game. So, you can think of a game where you like... You know, if we're playing a game where whoever names the bigger number is the winner-

Lex Fridman 8:11 Mm-hmm.

Noam Brown 8:12 ... there's no Nash equilibrium to that game.

Lex Fridman 8:13 17.

Noam Brown 8:14 Yeah, exactly. 18. I thought you'd beat me.

Lex Fridman 8:16 You win again.

Noam Brown 8:17 (laughs)

Lex Fridman 8:17 You're good at this.

Noam Brown 8:18 I played a lot of games.

Lex Fridman 8:21 (laughs) Okay. Uh, so that's th-... And then the zero-sum aspect. The zero-sum-

Noam Brown 8:24 The zero, zero-sum aspect... So, there exists a Nash equilibrium in non-two-player zero-sum games as well. And by the way, just to clarify what I mean by two-player zero-sum. I mean, there's two players and whatever one player wins, the other player loses. So, if we're playing poker and I win $50, that means that you're losing $50. Now, outside of two-player zero-sum games, there still exists Nash equilibria, but they're not as meaningful because, you know, you can think of a game like Risk. Y- If everybody else at the, on the board decides to team up against you and take you out, there's no perfect strategy you can play that's gonna guarantee that you win there. There's just nothing you can do. So, outside of two-player zero-sum games, there's no guarantee that you're going to win by playing a Nash equilibrium.

Lex Fridman 9:07 Have you ever tried to model in the other aspects of the game, which is like, the pleasure you draw from playing the game, and then if you're a professional poker player, if you're exciting even if you lose, uh, the, you know, the money you would get from the attention you get to the sponsors and all that kind of stuff. Is that... That'd be a fun thing to model, to model in. Or does that be- make it sort of super complex to, to include the human factor in this, in its full complexity?

Noam Brown 9:37 Uh, I think you bring up a couple good points there. So, I think a lot of professional poker players, I mean, they get a huge amount of money, not from the actually playing poker, but from the sponsorships and having a personality that people want to tune in and watch. That, that's a big, that's a big way to, to make a name for yourself in poker.

Lex Fridman 9:53 I just wonder from an AI perspective, if, if you create, and we'll talk about this more, maybe a- AI system that also talks trash and all that kind of stuff, that, that becomes part of the function to maximize. So, it's not just, um, optimal poker play. Maybe sometimes you want to be chaotic. Maybe sometimes you want to be sub-optimal and you lose, um, the, the chaos. And maybe sometimes you want to be overly aggressive because pe- the audience loves that.

Noam Brown 10:21 Yeah.

Lex Fridman 10:21 That'd be fascinating.

Noam Brown 10:23 I think, I think what you're getting at here is that there's a difference between making an AI that wins a game and an AI that's fun to play with, right?

Lex Fridman 10:28 Yeah.

Noam Brown 10:28 Yeah. And-

Lex Fridman 10:29 Or fun to watch.

Noam Brown 10:30 Or, or fun to watch, yeah.

Lex Fridman 10:30 So, those are all different things. Fun to play with and fun to watch.

Noam Brown 10:33 Yeah. And I think, you know, I've, I've heard, uh, talks from like, game designers and, and they say like, you know, people that work on AI for actual recreational games that people play. And they say, "Yeah, there's a big difference between trying to make an AI that actually wins and..." You know, you look at a game like Civilization. Um, the way that the AIs play is not optimal for trying to win. They're, they're playing a different game. They're trying to have personalities, they're trying to be fun and engaging. Um, and that makes for a better game. Yeah.

Lex Fridman 11:01 And we also talk about NPCs. I just talked to Todd Howard, who is the, the creator of Fallout and the Elder Scrolls series, and, um, Starfield, the new game coming out, and the cr- creator of what I think is the greatest game of all time, which is Skyrim, and the NPCs there. The AI that governs that whole game is very interesting, but the NPCs also are super interesting, and considering what language models might do to NPCs in an open-world RPG, role-playing game, is super exciting.

Noam Brown 11:31 Yeah. Honestly, I'm... I think this is like, one of the first applications where we're gonna see like, real consumer interaction with large language models. Um, I, I guess what Sky- um, Elder Scroll VI is in development now. They're probably like, pretty close to finishing it, but I, I would not be surprised at all if Elder Scroll VII was using large language models for their NPC.

Lex Fridman 11:49 No, they're, they're no- they're... I mean, I'm not saying anything.

Noam Brown 11:52 (laughs)

Lex Fridman 11:52 I'm not saying anything.

Noam Brown 11:53 Okay.

Lex Fridman 11:54 (laughs)

Noam Brown 11:54 This is me speculating, not you. (laughs)

Lex Fridman 11:55 No. But, but there's... They're just releasing the Starfe- Field game. They, they do one game at a time.

Noam Brown 12:00 Yeah.

Lex Fridman 12:00 And so, uh, whatever it is, whenever the date is, I don't know what the date is, calm down. Uh, but it would be, I don't know, like, uh, 2024, '25, '26. So, it's actually very possible they would include language models.

Noam Brown 12:14 I was listening to this, um, this talk by a gaming executive, uh, when I was in grad school, and one of the questions that-... a person in the audience asked is, "Why are all these games so focused on fighting and killing?" And the person responded that it's just so much harder to make an AI that can talk with you and cooperate with you than it is to make an AI that can fight you. (chair creaks) And I think once this technology develops further and you can have a, you can reach a point where, like, not every single line of dialogue has to be scripted, it unlocks a lot of potential for new kinds of games, like much more, like, positive interactions that are not so focused on fighting, and I'm really looking forward to that.

Lex Fridman 12:51 (laughs) Well, it might not be positive. It might be just drama.

Noam Brown 12:54 (laughs) Or that too. Yeah.

Lex Fridman 12:54 So you would, you, you would be in the, like, a Call of Duty game, and instead of doing the shooting, you'll just be hanging out and, like, arguing with an AI about, like, um, like, passive-aggressive, and then you won't be able to sleep that night. You have to return and ar- continue the argument-

Noam Brown 13:07 Mm-hmm.

Lex Fridman 13:07 ... that you were, uh, (laughs) emotionally hurt. Uh...

Noam Brown 13:11 (laughs)

Lex Fridman 13:12 I mean, yeah, I, I think that's actually an exciting world. Whatever, whatever is the drama, the chaos that we love, the push and pull of human connection, I think it's possible to do that in the video game world.

Noam Brown 13:21 Hm.

Lex Fridman 13:22 And it, I think you could be messier and make more mistakes in the video game world, which is why it would be a nice place. And, and also, it doesn't have a deep of a, as deep of a real psychological impact because inside video games, it's kinda understood that you're in a not a real world. So, whatever crazy stuff AI does, we have some flexibility to play. Just like with a game of Diplomacy, it's a game. This is not real geopolitics, not real war. It's a, it's a game. So you can, you can have a little bit of fun, um, a little bit of chaos. Okay, back to Nash equilibrium. (laughs) Uh, how do we find the Nash equilibrium?

Noam Brown 13:59 All right. So, there's different ways to find a Nash equilibrium. So, um, the way that we do it is with this process called self-play. Um, basically, we have this algorithm that starts by playing totally randomly, and it learns how to play the game by playing against itself. So, um, it will start playing the game totally randomly and then it, you know, if it's playing poker, it'll eventually, like, get to the end of the, end of the game and make $50. And then it will, like, review all of the decisions that it made along the way and say, "What would've happened if I had chosen this other action instead? You know, if I had raised here instead of called, um, what would the other player have done?" And because it's playing against a copy of itself, it's able to do that counterfactual reasoning. So it can say, "Okay, well, if I took this action and the other person takes this action and then I take this action and eventually I make $150 instead of 50." And so it updates the regret value for that action. Regret is basically like how much does it regret having not played that action in the past? And when it encounters that same situation again, it's going to pick actions that have higher regret with higher probability. Now, it'll just keep simulating the games this way. It'll keep, uh, you know, accumulating regrets for different situations. Um, and in the long run, if you pick actions that have higher regret with higher probability in the correct way, it's proven to converge to a Nash equilibrium.

Lex Fridman 15:24 Even for super complex games? Even for imperfect information games?

Noam Brown 15:28 It's true for all games. It's true for, it's true for chess. It's true for poker. It's particularly useful for poker.

Lex Fridman 15:33 So, this is the, the method of counterfactual regret minimization?

Noam Brown 15:36 This is counterfactual regret minimization.

Lex Fridman 15:37 But that doesn't have to do with self-play. It has to do with just... An- a- any... If you follow this kind of process, self-play or not, you will be able to arrive at an optimal set of actions?

Noam Brown 15:48 So this counterfactual regret minimization is a kind of self-play. It's a principled kind of self-play that's proven to converge to Nash equilibria, even in imperfect information games. Now, you can have other forms of self-play, and people use other forms of self-play for perfect information games, um, where you have more flexibility. The algorithm doesn't have to be as theoretically sound, um, in order to converge to, to that class of games, because there's, uh, it's a simpler setting.

Lex Fridman 16:12 Sure. So I, I kind of, in, in my brain, the word self-play has mapped in neural networks, but we're speaking something bigger than just neural networks. It could be anything. Uh, uh, th- the self-play mechanism, it ju- is just the mechanism of a system playing itself.

Noam Brown 16:26 Exactly, yeah. Self-play is not tied specifically to neural nets. It's, it's a kind of reinforcement learning basically.

Lex Fridman 16:31 Okay.

Noam Brown 16:32 And I would also say, this process of, like, trying to reason, "Oh, what would the value have been if I had taken this other action instead?" This is very similar to how humans learn to play a game like poker, right? Like, you probably played poker before, and-

Lex Fridman 16:44 Mm-hmm.

Noam Brown 16:44 ... with your friends, you probably ask, like, "Oh, would you have called me if I raised there?"

Lex Fridman 16:48 Right.

Noam Brown 16:49 You know, and that's, that's a person trying to do the same kind of, like, learning from a counterfactual that the AI is doing.

Lex Fridman 16:54 Okay. And if you do that at scale, you're gonna be able to learn an optimal policy.

Noam Brown 16:59 Yeah. Now, where the neural nets come in, I said, like, okay, if it's in that situation again, then it will choose the action that has, has higher regret. Now, the problem is that poker is such a huge game, you know, I think in No-Limit Texas Hold'em, the version that we were playing has 10 to the 161 different decision points, which is more than the number of atoms in the universe squared.

Lex Fridman 17:18 That's heads-up?

Noam Brown 17:19 That's heads-up. Yeah.

Lex Fridman 17:20 10 to the 161, you said?

Noam Brown 17:21 Yeah. I mean, it depends on the number of chips that you have, the stacks and everything, but, like, the version that we were playing was 10 to the 161.

Lex Fridman 17:27 Which I assume would be a somewhat simplified version anyway, 'cause the... I bet there's some, like, step function you had for, like, bets. Right?

Noam Brown 17:36 Oh, no, no, no. That's, that's... I'm saying, like, we played the, the full game. You can bet whatever amount you want.

Lex Fridman 17:39 Okay.

Noam Brown 17:39 And the bot maybe was constrained in, like, what it considered for bet sizes, but the-

Lex Fridman 17:43 Yeah.

Noam Brown 17:43 ... the person on the other side could bet whatever they wanted.

Lex Fridman 17:45 Yeah. I mean, 161 plus or minus 10 doesn't matter. (laughs)

Noam Brown 17:49 Yeah, 161, yeah. (laughs) Um... And so-

Lex Fridman 17:52 (laughs) Yeah.

Noam Brown 17:52 ... the, the way neural nets help out here is, you know, you don't have to run into the same exact situation, 'cause that's never gonna happen again. The odds of you running into the same exact situation are pretty slim. But if you run into a similar situation, then you can generalize from other states that you've been in that kind of look like that one and you can say like, "Well, these other situations, I had high regret for this action, and so maybe I should play that action here

Noam Brown 18:12 as well."

Lex Fridman 18:12 Which is the more complex game, chess or poker? Or Go or poker? Do you know?

Noam Brown 18:19 That is, that is a controversial question.

Lex Fridman 18:21 Okay.

Noam Brown 18:21 I'm gonna-

Lex Fridman 18:22 Oh, is, like, somebody screaming on Reddit right now, "It depends on which subreddit you're on"?

Noam Brown 18:25 Yeah.

Lex Fridman 18:25 "Is it chess or is it poker?"

Noam Brown 18:27 Yeah. I'm sure, like, David Silver is gonna get really angry at me.

Lex Fridman 18:29 Yeah.

Noam Brown 18:29 Uh, I'll, I'll say, I'm gonna say poker actually, and I think for a couple reasons. Um-

Lex Fridman 18:33 They're not here to defend themselves.

Noam Brown 18:35 (laughs) So, first of all, you have the imperfect information aspect. And so it's, um, i-We, we can go into that, but like once you introduce imperfect information, uh, things get much more complicated.

Lex Fridman 18:49 So we should say, maybe you can describe what is seen to the players, what is not seen, um, in the game of Texas Hold'em?

Noam Brown 18:58 Yeah. So Texas Hold'em, you get two cards face down that only you see. Um, and so that's the hidden information of the game. The other players also all get two cards face down that only they see. Um, and so you have to kind of as you're playing reason about like, "Okay, what do they think I have? What do they have? What do they think I think they have?" That kind of stuff. And, um, that's, that's kind of where bluffing comes into play, right? Because the fact that you can bluff, the fact that you can bet with a bad hand and still win is because they don't know what your cards are.

Lex Fridman 19:27 Right.

Noam Brown 19:28 And that's the, that's the key difference between a perfect information game like poker... uh, sorry, like chess and Go, um, and imperfect information games like poker.

Lex Fridman 19:36 This is what trash talk looks like. (laughs) The implied statement is, "The game I solved is much tougher."

Noam Brown 19:43 (laughs)

Lex Fridman 19:44 Uh, but yeah, so, uh, w- when you're playing... I'm just gonna do random questions here. So wha- when you're playing your opponent under imperfect information, is there some degree to which you're trying to estimate the range of hands that they have? Or is that not part of the algorithm? So how, uh... what are the different approaches to the imperfect information game?

Noam Brown 20:06 So the key thing to understand about why imperfect information makes things difficult-

Lex Fridman 20:10 Mm-hmm.

Noam Brown 20:11 ... is that you have to worry not just about which actions to play, but the probability that you're going to play those actions. So you think about, um, rock-paper-scissors for example. Rock-paper-scissors is an imperfect information game, um-

Lex Fridman 20:24 Right.

Noam Brown 20:24 ... because you don't know what I'm about to throw.

Lex Fridman 20:26 I, I do, but yeah, usually not. Yeah.

Noam Brown 20:28 Yeah. And so you can't just say like, "Oh, I'm just going to throw a rock every single time-"

Lex Fridman 20:32 Mm-hmm.

Noam Brown 20:32 ... because the other person is gonna figure that out and notice a pattern, and then suddenly you're gonna start losing.

Lex Fridman 20:37 Mm-hmm.

Noam Brown 20:37 And so you don't just have to figure out, like, which action to play, you have to figure out the probability that you play it. And really importantly, the value of an action depends on the probability that you're gonna play it. So if you're playing rock every single time, that value is really low, but if you're never playing rock, you play rock like 1% of the time, then suddenly the, the other person's probably gonna be throwing scissors. And when you throw a rock, the value of that action is gonna be really high. Now, you take that to poker, what that means is the value of bluffing, for example, if you're the kind of person that never bluffs and you have this reputation as somebody that, that never bluffs and suddenly you bluff, there's a really good chance that that bluff is gonna work and you're gonna make a lot of money. On the other hand, if you got a reputation, like if they've seen you play for a long time and they see, oh, you're the kind of person that's bluffing all the time, when you bluff, they're not gonna buy it and they're gonna call you down. You're gonna lose a lot of money. And that... finding that balance of how often you should be bluffing is, uh, the key challenge of a game of poker. And, um, you contrast that with a game like chess, it doesn't matter if you're opening with the queen's gambit 10% of the time or 100% of the time, the value... the expected value is the same. So, um, so that's, that's why we need these algorithms that understand not just we have to figure out what actions are good, but the probabilities. We need to get the exact probabilities correct. And that's actually when we created the bot, Libratus... Libratus means balanced because the algorithm that we designed was designed to find that right balance of how often it should play each action.

Lex Fridman 22:10 The balance of how often in the key sort of branching is to bluff or not to bluff.

Noam Brown 22:16 It's-

Lex Fridman 22:16 Is that, is that a... is that a good crude simplification of the major decision in poker?

Noam Brown 22:21 It's a good simplification. I think that's like the main tension, but it's, it's not just how often to bluff or not to bluff, it's like how often should you bet in general? How often should you... what, what kind of bet should you make? Um, should you bet big or should you bet small and with which... with, with which hands? Uh, and so this is where the idea of a... of a range comes from.

Lex Fridman 22:40 Yeah.

Noam Brown 22:40 Because when you are bluffing with a particular hand in a particular spot, you don't want there to be a pattern for the other person to pick up on. You don't want them to figure out, "Oh, whenever this person is in this spot, they're always bluffing." And so you have to reason about, "Okay, would I also bet with a good hand in this spot?" You wanna be unpredictable. So you have to think about, "What would I do if I had this different set of cards?"

Lex Fridman 23:04 Is there explicit estimation of like a theory of mine that the other person has about you or is that just a emergent thing that happens?

Noam Brown 23:14 The way that the bots handle it, that, that are really successful, they have an explicit theory of mind. So they're explicitly reasoning about what are... "What's the common knowledge belief? What is... what do you think I have? What do I think you have? What do you think I think you have?" Um, it's explicitly reasoning about that.

Lex Fridman 23:32 Is there multiple yous there? So maybe that's jumping ahead to six players, but I... is there a stickiness to the person to... so it's an iterative game, you're playing the same person. There is... there's a stickiness to that, right? You're gathering information as you play. It's not every g-... every, uh, every hand is a new hand. Is there, um, a continuation in terms of estimating what kind of player I'm facing here?

Noam Brown 23:59 That's a good question. So you could approach the game that way. The way that the bots do it, they don't... and the way that humans approach it also, expert human players, the way they approach it is to basically assume that you know my strategy. So I'm going to try to pick a strategy where even if I were to play it for 10,000 hands and you could figure out exactly what it was, you still wouldn't be able to beat it. Basically what that means is I'm trying to approximate the Nash equilibrium. I'm trying to be perfectly balanced because if, if I'm playing the Nash equilibrium, even if you know what my strategy is, like I said, I'm still unbeatable in expectation. So, so that's what... that's what the bot aims for and that's actually what a lot of expert poker players aim for as well, to start by playing the Nash equilibrium and then maybe if they spot weaknesses in the way you're playing, then they can deviate a little bit to take advantage of that.

Lex Fridman 24:47 They aim to be unbeatable in expectation. Okay.

Lex Fridman 24:50 So, who's the greatest poker player of all time and why is it Phil Hellmuth? So, this is for Phil. Uh, so he's known, um, at least in part for maybe playing sub-optimally, and he stills wins a lot. It's a bit chaotic. So, maybe can you speak from an AI perspective about the genius of his madness or the madness of his genius? So, playing sub-optimally, playing chaotically, um, as a way to make it hard to pin down about what your strategy is.

Noam Brown 25:23 So, okay. The thing that I should explain first of all with like Nash equilibrium, it doesn't mean that it's predictable. The whole point of it is that you're trying to be unpredictable. Now, I think when somebody like Phil Hellmuth might be really successful-

Lex Fridman 25:36 Right.

Noam Brown 25:37 ... is not in being unpredictable, but in being able to, um, take advantage of the other player and figure out where they're being predictable or guiding the other player into thinking that you have a certain weaknesses and then un- and then understanding how they're going to change their behavior. They're going to deviate from a Nash equilibrium style of play to try to take advantage of those perceived weaknesses and then counter exploit them. So, you kind of get into the mind games there.

Lex Fridman 26:02 So, you think about at least heads up poker as a, as a dance between two agents? I guess are you playing the cards or are you playing the, the player?

Noam Brown 26:10 So, this, this gets down to a big argument in the poker community and the academic community. For a long time, there was this debate of like what- what's called GTO, game theory optimal poker-

Lex Fridman 26:20 Right.

Noam Brown 26:20 ... or exploitative play. And, um, up until about like 2017 when we did the Libratus match, I think actually exploitative play had the advantage. A lot of people were saying like, "Oh, this whole idea of game theory, it's just nonsense. And if you really want to make money, you got to like look into the other person's eyes and read their soul and figure out what cards they have." But what happened was people started adopting the game theory optimal strategy, um, and they were making good money. And they weren't trying to adapt so much to the other player. They were just trying to play the Nash equilibrium. And then what really solidified it, I think, was the Libratus, the Libratus match where we played our bot against four top heads up no limit hold 'em poker players, and the bot wasn't trying to adapt to them. It wasn't trying to exploit them. It wasn't trying to do these mind games. It was just trying to approximate the Nash equilibrium, and it crushed them. I think, you know, it... We were playing for $50, $100 blinds, and over the course of about 120,000 hands, it made close to $2 million in-

Lex Fridman 27:21 120,000 hands?

Noam Brown 27:22 120,000 hands in-

Lex Fridman 27:24 A- against humans?

Noam Brown 27:24 Yeah. And this was, this was fake money, to be clear. So, there was real money at stake. There was $200,000-

Lex Fridman 27:28 First of all-

Noam Brown 27:29 Yeah.

Lex Fridman 27:29 ... all money is fake, but, um, that's a, that's- that's a different conversation. Um, we give it meaning. Uh, it's an, it's a, it's- it's a phenomena that gets meaning from our, uh, complex psychology as a human civilization. Um, it's emergent from the collective intelligence of the human species. But that's not what you mean. You mean like there's literally you can't, (laughs) you can't buy stuff with it. Okay. Can you actually, uh, step back and take me through that, um, competition?

Noam Brown 27:55 Yeah. Okay. So, when I was in grad school, um, there was this thing called the Annual Computer Poker Competition-

Lex Fridman 28:02 Mm-hmm.

Noam Brown 28:02 ... where every year all the different research labs that were working on AI for poker would get together. They would make a bot. They would play them against each other. Uh, and we made a bot that actually won the, um, 2014 competition, the 2016 competition, uh, and so we decided we're gonna take this bot, build on it, and play against real top professional heads up no limit Texas hold 'em poker players. So, we invited four of the world's best, um, players in this specialty, and we challenged them to 120,000 hands of poker over the course of 20 days. Um, and we had $200,000, $200,000 in prize money at stake where it would basically be divided among them depending on how well they did relative to each other. So, we wanted to have some incentive for them to play their best.

Lex Fridman 28:47 Did you have a confidence, 2014, '16, that this is even possible? How much doubt was there?

Noam Brown 28:54 So... And we did a competition actually in 2015 where we also played against professional poker players and the bot lost by- by a pretty sizable margin actually. Now, there were some big improvements from 2015 to 2017 and so-

Lex Fridman 29:06 Can you speak to the improvements? Is it computational in nature? Is it the algorithm, the- the methods? What?

Noam Brown 29:10 It was, it was really an algorithmic approach. That- that was the difference. So, 2015, it was much more focused on trying to come up with a strategy upfront, like trying to solve the entire game of poker, like and then just have a look up table where you're saying like, "Oh, I'm in this situation. What's the strategy?"

Lex Fridman 29:29 Mm-hmm.

Noam Brown 29:29 Um, the approach that we took in 2017 was much more search based. It was trying to say, "Okay, well, let me in real time try to compute, uh, a much better strategy than what I had pre-computed, um, by playing against myself during self play."

Lex Fridman 29:42 What is the search space for poker? What are you searching over? What's that look like? There's different actions like raising, calling. Yeah, what are the actions? Um, is it just a search over actions?

Noam Brown 29:59 So, in a game like chess, the- the search is like, "Okay, I'm in this chess position and I can like, you know, move these different pieces and see where things end up." In poker, what you're searching over is the actions that you can take for your hand, the probabilities that you take those actions, and then also the probabilities that you take other actions with other hands that you might have.

Lex Fridman 30:19 Right.

Noam Brown 30:19 Um, and, and that's kind of like, um, hard to wrap your head around, like why are you searching over these like other hands that you might have and like trying to figure out what you would do with those hands?

Lex Fridman 30:29 Mm-hmm.

Noam Brown 30:30 Um, and the idea is, is again, you- you wanna, you wanna always be balanced and unpredictable. And so if your search algorithm is saying like, "Oh, I want to raise with this hand," well, in order to know whether that's a good action... Like let's say it's a bluff, you know? Let's say you have a bad hand and you're saying like, "Oh, I- I think I should be betting here with this really bad hand and bluffing," well, that on- that's only a good action if you're also...... betting with a strong hand. Otherwise, it's an obvious bluff.

Lex Fridman 30:57 So if your action, in some sense, maximizes your imp- imp- unpredictability, so that action could be mapped by your opponent to a lot of different hands, then that's a good action.

Noam Brown 31:07 Basically what you want to do is put your opponent into a tough spot. So, you want them to always have some doubt, like, "Should I call here? Should I fold here?" And if you are raising in the appropriate balance between bluffs and good hands, then you're putting them into that tough spot. And so that's what we're trying to do. We're always trying to search for a strategy that would put the opponent into a difficult position.

Lex Fridman 31:26 Can you give a metric that you're trying to maximize or minimize? Does this have to do with the regret thing when we were talking about in terms of putting your opponent in a maximally tough spot?

Noam Brown 31:37 Yeah. Ultimately what you're trying to maximize is your expected winnings, like your expected value, the amount of money that you're gonna walk away from, assuming that your opponent was playing optimally in response. So, you're gonna assume that your opponent is- is also playing, um, like as- as well as possible a Nash equilibrium approach, because if they're not, then you're just gonna make more money, right? Like anything that deviates... Like, by definition, the Nash equilibrium is the strategy that does the best in expectation, and so if you're deviating from that then you're just... They're gonna lose money and since it's a two player zero sum game, that means you're gonna make money.

Lex Fridman 32:12 So there's not an explicit like objective function that maximizes the toughness of the spot they're put in. You're always... This is from like a self-play reinforcement learning perspective, you're just trying to maximize winnings, and the rest is implicit.

Noam Brown 32:27 That's right, yeah. So, we're... What we're actually trying to maximize is the expected value given that the opponent is playing optimally in response to us. Now, in practice what that ends up looking like is it's putting the opponent into difficult situations where-

Lex Fridman 32:39 Got it.

Noam Brown 32:39 ... there's no obvious decision to be made.

Lex Fridman 32:41 So the- the system doesn't know anything about the difficulty of the situation?

Noam Brown 32:46 Not at all. Doesn't care.

Lex Fridman 32:47 Okay.

Noam Brown 32:47 Yeah.

Lex Fridman 32:47 All right. In my head it was getting excited whenever it's making the other, the opponent sweat. Okay, so you're... In 2015 you didn't do as well, so what's the journey from that to a system that, in your mind, could have a chance?

Noam Brown 33:00 So 2015, we- we got... We beat- beat pretty badly, and we actually learned a lot from that competition. And in particular, you know, what became clear to me is that the way the humans were approaching the game was very different from how the bot was approaching the game. The bot would not be doing search, it would just be trying to compute. You know, it would do like months of self-play. It would just be playing against itself for months, but then when it's actually playing the game it would just act instantly. Um, and the humans, when they're in a tough spot, they would sit there and think for sometimes even like five minutes about whether they're going to call or fold a hand. Um, and it became clear to me that that's... That there's a good chance that that's what missing, that's what's missing from our bot. So I actually did some, um, initial experiments to try to figure out how much of a difference does this actually make, and the difference was huge.

Lex Fridman 33:49 As a signal to the human player how long you took to think?

Noam Brown 33:52 No, no, no. I'm not saying that there were any timing tells. I was saying when the human... Like, the bot would always act instantly. It wouldn't try to come up with a better strategy in real time, um, over what it had pre-computed during training. Whereas the human-

Lex Fridman 34:05 Got it.

Noam Brown 34:05 ... like they have all this intuition about how to play, but they're also in real time leveraging their ability to- to think, to- to search, to plan, um, and coming up with an even better strategy than what- than what their intuition would say.

Lex Fridman 34:17 So you're saying that there's... You're doing... That's what you mean by you're doing search also?

Noam Brown 34:21 Yeah.

Lex Fridman 34:22 You have an... You have a intuition and search on top of that looking for a better solution?

Noam Brown 34:28 Mm-hmm. Yeah. That's- that's what I mean by search, that, um, instead of acting instantly, you know, a neural net usually gives you a response in like 100 milliseconds or something. It depends on the size of the- of the net. But if you can leverage extra computational resources, you can possibly get a much better outcome. And we did some experiments in small scale versions of poker and what we... what we found was that if you do a little bit of search, even just a little bit, it was the equivalent of making your, you know, your pre-computed strategy, like you could kind of think of it as your neural net 1,000 times bigger with just a little bit of search. And it just like blew away all of the research that we had been working on and trying to like scale up this like pre-computed solution. It was dwarfed by the benefit that we got from search.

Lex Fridman 35:19 Can you just linger on what you mean by search here? You're searching over a space of actions for your hand and for other hands. How are you selecting the other hands to search over? Is-

Noam Brown 35:32 So yeah.

Lex Fridman 35:33 ... randomly?

Noam Brown 35:34 No, it's all the other hands that you could have. So when you're playing no limit Texas hold 'em, you've got two face down cards, and so that's 52 choose 2, 1,326 different combinations. Now that's actually a little bit lower because there's face up cards in the middle and so you can eliminate those as well.

Lex Fridman 35:48 Okay.

Noam Brown 35:48 But you're looking at like around 1,000 different possible hands that you can have. And so when we're doing... When the bot's doing search, it's thinking explicitly there are these thousand different hands that I could have, there are these thousand different hands that you could have, let me try to figure out what would be a better strategy than what I've pre-computed for these hands and your hands.

Lex Fridman 36:07 Okay. So that search, how do you fuse that with what the neural net is telling you or what the- the- the trained system is telling you?

Noam Brown 36:19 Yeah. So, you kind of like... Where the train system comes in is- is the value, um, at the end.

Lex Fridman 36:25 Got it.

Noam Brown 36:25 So there's... Um, you only look so far ahead, you look like maybe, you know, one round ahead. So if you're on the flop you're looking to the start of the turn.

Lex Fridman 36:34 Mm-hmm.

Noam Brown 36:34 Um, and at that point you can use the pre-computed solution to figure out what are... What's the value here of like of this strategy?

Lex Fridman 36:43 Is it of a single action essentially in that spot you're getting a value or is it the value of the entire series of actions?

Noam Brown 36:52 Well, it's kind of both, um, because you're- you're trying to maximize the value for...... the hand that you have, but in the process, in order to maximize the value of the hand that you have, you have to figure out, "What would I be doing with all these other hands as well?" Um-

Lex Fridman 37:04 Okay. But you- are you in the search, always going to the end of the game?

Noam Brown 37:09 No. In Libratus, we did. Uh, so we only used search starting on the turn, and then we searched all the way to the end of the game.

Lex Fridman 37:17 The turn, the river. Uh, c-can we take it-

Noam Brown 37:21 Yeah. Yeah, yeah. Th-

Lex Fridman 37:21 ... t-through the terminology?

Noam Brown 37:22 Yeah. There's, there's four rounds of poker. So, there's the pre-flop, the flop, the turn, and the river. Uh, and so we would start doing search halfway through the game. Now, the first half of the game, that was all pre-computed. It would just act instantly. And then when it got to the halfway point, then it would always search to the end of the game. Now, we later improved this so it wouldn't have to search all the way to the end of the game. It would actually search, um, just a few moves ahead. Um, but that, that came later, and that drastically reduced the nu- the amount of computational resources that we needed.

Lex Fridman 37:51 But the moves, 'cause you can keep betting on top of each other, that's what you mean by moves? So like, that's where you don't just get one bet per turn of poker. You can have multiple arbitrary number of bets, right?

Noam Brown 38:02 Right. I'm trying to think like, I'm gonna bet, and then what are you gonna do in response? Are you gonna raise me or are you gonna call? And then if you raise, what should I do? So, it's reasoning about that whole process up until the end of the game, in the case of Libratus.

Lex Fridman 38:15 So for Libratus, what's the, the most number of re-raises have you ever seen? (laughs)

Noam Brown 38:20 Uh, you probably cap out at like five or something, because at that point, you're basically all in, you know?

Lex Fridman 38:26 I mean, i- is there like, uh, interesting patterns like that that you've seen that the game does? Like, you, you'll have like AlphaZero doing way more sacrifices than hu-humans usually do. Is there something, like Libratus was constantly re-raising or something like that that you've noticed?

Noam Brown 38:43 There was, there was something really interesting that we observed with Libratus. Um, so humans, when they're playing poker, they usually size their bets relative to the size of the pot. So, you know, if the pot has $100 in there, maybe you bet like $75 or s-

Lex Fridman 38:56 Mm-hmm.

Noam Brown 38:57 ... somewhere around there. Somewhere between like $50 and $100. Um, and with Libratus, we gave it the option to basically bet whatever it wanted. It, it was actually really easy for us to say like, "Oh, if you want, you can bet like 10 times the pot."

Lex Fridman 39:09 Mm-hmm.

Noam Brown 39:09 And we didn't think it would actually do that. It was just like, why not give it the option? And then during the competition, it actually started doing this. And by the way, this is like a very last minute decision on our part to add this option. And so we did not n- We, we did not think the bot would, would do this, and, uh, I was actually kind of worried when it did start to do this. Like, "Oh, is this a, is this a problem? Like, humans don't do this." Like-

Lex Fridman 39:28 Mm-hmm.

Noam Brown 39:29 "Is it screwing up?" Um, but it would put the humans into really difficult spots when it would do that, because, you know, you can imagine, like you have the second-best hand that's possible given the board, and you're thinking like, "Oh, you're in a really great spot here." And suddenly, the bot bets $20,000 into a, you know, $1,000 pot, and, and it's basically saying like, "I have the best hand or I'm bluffing." And you having the second-best hand, like now you got a really tough choice to make. And so the humans would sometimes think like five or 10 minutes about like, "What do you do? Should, should I call? Should I fold?" And, um, and when I saw the humans like really struggling with that decision, like that's when I realized like, "Oh, actually this is maybe a good thing to do after all."

Lex Fridman 40:09 And of course, the system doesn't know that it's making, again, like we said, that it's putting them at a tough spot. It's, it's, it's just that's part of the optimal, the, the game theory optimal.

Noam Brown 40:21 Right. From the bot's perspective, it's just, it's just doing the thing that's going to make it the most money. Um, and, but the fact that it's putting the humans in a difficult spot, like that's just, um, you know, a side effect of that. And this was, I think, the, the one thing... I mean, there were a few things that the humans walked away from, but this was the, the number one thing that the humans walked away from the competition saying like, "We need to start doing this."

Lex Fridman 40:43 Hmm.

Noam Brown 40:43 Um, and now these over-bets, what are called over-bets, have become really common in high-level poker play.

Lex Fridman 40:48 Have you ever talked to like somebody like Daniel Negreanu about this? He seems to be a student of the game.

Noam Brown 40:54 I did actually have a conversation with Daniel Negreanu once, yeah. I was, uh, I was visiting the Isle of Man to talk to poker stars about AI. Um, and Daniel Negreanu was there. We, and we had dinner together with, uh, some other people. And, um, yeah. He's, he was really interested in it. He mentioned that he was like, you know, excited about like learning from these AIs, um-

Lex Fridman 41:14 So, he wasn't scared. He was excited.

Noam Brown 41:16 He was excited. And, uh, and he ho- he honestly, he wanted to play against the bot. He thought he, he thought he had a decent chance of, of beating it. Um, I-

Lex Fridman 41:23 Yeah.

Noam Brown 41:23 ... I think he, you know... This was like, uh, uh, several years ago when I think it was like not as clear to everybody that, you know, the AIs were taking over.

Lex Fridman 41:32 Yeah.

Noam Brown 41:32 I think now people recognize that like if you're playing against, uh, a bot, there's like no chance that you have in a game like poker.

Lex Fridman 41:38 So consistently the bots will win. The bots of heads-up and in, in other variants too? So multi, multi, six-player Texas hold 'em, no-limit Texas hold 'em is the bots win?

Noam Brown 41:51 Yeah. That's the case. So I think there is some debate about like, is it true for every single variant of poker? I think, I think for every single variant of poker, if somebody really put in the effort, they can make an AI that would beat all humans at it. Um, we've focused on the most popular variants, so heads-up, no-limit Texas hold 'em, and then we followed that up with, um, with, uh, six-player poker as well, where we managed to, uh, make a bot that beat expert human players. And I think even there now, uh, it's pretty clear that humans don't stand a chance.

Lex Fridman 42:20 See, I would love to hook up an AI system that looks at EEG, like how, like actually tries to optimize the toughness of the spot it puts a human in. And I, I would, I would love to see how different is that from the game theory optimal? So, you try to maximize the heart rate of the human player.

Noam Brown 42:39 Mm-hmm.

Lex Fridman 42:39 (laughs) Like the freaking out over a long period of time. I wonder if there's going to be different strategies that emerge, uh, that are close in terms of effectiveness 'cause it, something tells me you could still be, um, achieve superhuman level of performance by just making people sweat.

Noam Brown 42:58 I feel like that there's a good chance that that is the case, yeah. If you're able to see, like... That- th- it's like a, it's like a decent proxy for score, right?

Lex Fridman 43:06 Right.

Noam Brown 43:06 Um, and this is actually, like, the- the common poker wisdom when they're tell- where they're teaching players. Before there were bots and they were trying to teach people how to play poker, they would say, like, "The key to the game is to put your opponent into difficult spots." It's a good, um, a good estimate for if you're making the right decision.

Lex Fridman 43:21 So, what else can you say about the fundamental role of search in poker? And maybe if you can also relate it to chess and Go, in these games, um, what's the role of search to solving these games?

Noam Brown 43:37 Yeah. I think a lot of people under... This is true for the general public and I think it's true for the AI community, a lot of people underestimate the importance of search for these kinds of game AI results. Um, an example of this is, uh, TD Gammon that came out in 1992. This was the- the first real instance of a neural net being used in a game AI. It's a landmark achievement. It was actually the inspiration for AlphaZero, and it used search, it used two-pie search to figure out its next move. You got Deep Blue. There, it was very heavily focused on search, um, looking many, many moves ahead, farther than any human could. And that was key for why it won. And then even with something like AlphaGo. I mean, AlphaGo is commonly hailed as a- a landmark achievement for neural nets, and it is, but there's also this huge component of search, Monte Carlo tree search to AlphaGo, that was key, absolutely essential for the AI to be able to beat top humans. Um, I think a good example of this is you look at the latest versions of Alpha- of AlphaGo, like, uh, it was called AlphaZero, um, and there's this metric called Elo rating where you can compare different humans and you can compare bots to humans. Now, a top human player is around 3,600 Elo, maybe a little bit higher now. Um, AlphaZero, the strongest version, is around 5,200 Elo. But if you take out the search that's being done at test time and... But by the way, what I mean by search is the planning ahead, the- the thinking of like, "Oh, if I move my... If I place this stone here and then he does this," and then you look like five moves ahead and you see like what the board state looks like, um, that's what I mean by search. If you take out the search that's done during the game, the Elo rating drops to around 3,000. So even today, what, seven years after AlphaGo, if you take out the Monte Carlo tree search that's being done at one playing against a human, the bots are not superhuman. Nobody has made a raw neural net that is superhuman in Go.

Lex Fridman 45:39 That's worth lingering on. That's- that's quite profound. So, uh, without search, that just means looking at the next move and saying, "This is the best move." So having a function that estimates accurately what the best move is-

Noam Brown 45:54 That's right.

Lex Fridman 45:54 ... without search.

Noam Brown 45:56 Yeah. And all these bots they have the- the- what's called a policy network where it will tell you, "This is what the neural net thinks is the next best move." Um, and it- it's kind of like a- the intuition that a human has. You know, the- the human looks at the board and- and any, uh, Go or chess master will be able to tell you like, "Oh, instantly here's what I think the right move is." Um, and the bot is able to do the same thing. But just like how a human grandmaster can make a better decision if they have more time to think, when you add on this Monte Carlo tree search, the bot is able to make a better decision.

Lex Fridman 46:30 Yeah. I mean, of course a human is doing something like search in their brain, but it's not... Uh, I hesitate to draw a hard line, but it's not like, uh, Monte Carlo tree search. It's more like sequential language model generation. So it's like a diff- it's a... The neural network is doing the searching and I- I wonder what the human brain is doing in terms of searching, because you're doing that like computation.

Noam Brown 46:55 Mm-hmm.

Lex Fridman 46:56 A human is computing. They have intuition, they have gut, they have a really strong ability to estimate, you know, amongst the top players of what is good and not position without calculating all the details, but they're still doing search in their head, but it's a different kind of search. Have you ever thought about like what is the difference between the human, the search that the human is performing versus what computers are doing?

Noam Brown 47:21 I have thought a lot about that and I think it's a really important question. So, the AI in Alpha and Alphas... In AlphaGo or any of these Go AIs, they're all doing Monte Carlo tree search, which is a particular kind of search. And it- it's actually a- a symbolic tabular search. It- it uses the neural net to guide its search, but it isn't actually like full- full-on neural net. Now, that kind of search is very successful in these kinds of like perfect information board games like chess and Go. But if you take it to a game like poker, for example, it doesn't work. It- it can't- it can't understand the concept of hidden information. It doesn't understand the balance that you have to strike between like the amount that you're raising versus the amount that you're calling.

Lex Fridman 48:02 Mm.

Noam Brown 48:02 And in every one of these games, you see a different kind of search. And the human brain is able to plan for all these different games in a very general way. Um, now I think that's one thing that we're- we're missing from AI today and I think it's a really important missing piece, the ability to plan and reason more generally, um, across a wide variety, uh, of different settings.

Lex Fridman 48:23 In a way where the general reasoning makes you better at each one of the games.

Noam Brown 48:28 Yeah.

Lex Fridman 48:28 Not worse.

Noam Brown 48:29 Yeah. So you can kind of think of it as like neural nets today, they'll give you... Like transformers, for example, are super general. But, you know, they'll give you... It'll- it'll output an answer in like 100 milliseconds, and if you tell it like, "Oh, you've got five minutes to give me a decision, you know, feel free to take more time to make a better decision," it's not gonna know what to do with that. Um, but a human, if you're playing a game like chess, they're gonna give you a very different answer depending on if you say, "Oh, you've got 100 milliseconds or you've got five minutes."

Lex Fridman 48:57 (sighs) Yeah. There... I mean, there've been people who have started using, right, transformers with language models, like the... in an iterative way that does improve the answer, or, like, showing the work kind of-

Noam Brown 49:06 They've got this... yeah.

Lex Fridman 49:07 ... kind of idea?

Noam Brown 49:08 Yeah. They got this thing called chain of thought reasoning.

Lex Fridman 49:09 Mm-hmm.

Noam Brown 49:10 And that's, I think, um-

Lex Fridman 49:11 Super promising, right?

Noam Brown 49:12 Yeah. I think- and I think it's a good step in- in the right direction. Um, I- I would kind of, like, say it's similar to Monte Carlo rollouts in- in a game like chess. There's a kind of search that you can do where you're saying, like, "I'm gonna roll out my intuition and see, like... without really thinking, you know, what are the better decisions I could make farther down the path, um, what would I do if I just acted according to intuition for the next 10 moves?" Um, and that gets you an improvement, but I think that there's much, uh, much richer kinds of- of planning that we could do.

Lex Fridman 49:42 So when Libratus actually beat the poker players, wha- what did that feel like? What was that... I mean, actually on that day, what were you feeling like? Were you- were you nervous? I mean, poker was one of the games that you thought, like, is not gonna be solvable 'cause of the human factor, so at least in the narratives we tell ourselves, the human factor is so fundamental to the game of poker.

Noam Brown 50:05 Yeah. The Libratus competition was super stressful for me. Um, y- also, I mean, I was working on this for, like, basically continuously for a year leading up to the competition. I mean, for me, it became, like, very clear, like, okay, this is the search technique. This is the approach that we need, and then I spent a year working on this pretty much, like, nonstop. Yeah.

Lex Fridman 50:22 Oh, can we actually get into details, like what programming languages is it written in? What's some interesting, uh, implementation details that are, like, fun/painful?

Noam Brown 50:33 Yeah. So one of the interesting things about Libratus is that we had no idea what the bar was to actually beat top humans.

Lex Fridman 50:39 Yeah.

Noam Brown 50:39 Uh-

Lex Fridman 50:39 Right.

Noam Brown 50:39 ... we could play against, like, our prior bots, and that kind of gives us some sense of like, are we making progress? Are we going in the right direction? Uh, but we had no idea, like, what the bar actually was. And so, we threw a huge amount of resources at trying to make the strongest bot possible.

Lex Fridman 50:53 Mm-hmm.

Noam Brown 50:53 So, we used C++. It was parallelized. We were using, I think, like, a thousand CPUs, um, maybe- maybe more actually. Um, and, you know, today that sounds like nothing, but for a grad student back in 2016, that was a- a huge amount of resources kind of thing.

Lex Fridman 51:06 Well, it's still a lot for even any grad student today.

Noam Brown 51:09 Yeah.

Lex Fridman 51:09 It's still tough to- to get or even to- to allow yourself to think in that- uh, in terms of scale at CMU, at MIT, anything like that.

Noam Brown 51:19 Yeah. And, you know, talking about terabytes of memory, um, so it was a very parallelized, um... and it had to be very fast too, because the more games that you could simulate, uh, the stronger the bot would be.

Lex Fridman 51:30 So, is there some, like, John Carmack-style, like, efficiencies you had to come up with, like a- an efficient way to represent the hand, all that kind of stuff?

Noam Brown 51:40 There were all- all sorts of optimizations that I had to make to try to get this thing to run as fast as possible. They were like, how do you minimize the latency? How do you, like, you know, package things together so that, like, you minimize the amount of communication between the different nodes?

Lex Fridman 51:53 Mm-hmm.

Noam Brown 51:53 Um, how do you, like, optimize the algorithms so that you can, you know, try to squeeze out more and more from the game that you're actually playing? All- all these kinds of different decisions that- that I, you know, had to make.

Lex Fridman 52:03 Uh, just a fun question. What- what IDE did you use? What, uh... for- for C++ at the time?

Noam Brown 52:10 Uh, I think- I think I used, uh, Visual Studio actually. Yeah.

Lex Fridman 52:12 Okay.

Noam Brown 52:13 Yeah.

Lex Fridman 52:13 Is that still carried through to today?

Noam Brown 52:15 VS Code is- is what I use today.

Lex Fridman 52:17 Yeah.

Noam Brown 52:17 It seems like it's pretty popular.

Lex Fridman 52:17 It's the community basically converged on. Okay, cool. So, you got- you got this super optimized C++ system, and then you show up to the day of competition-

Noam Brown 52:28 Yeah.

Lex Fridman 52:29 ... humans versus machine. Um, how did it feel throughout the day?

Noam Brown 52:34 Super stressful. Um, I mean, I thought going into it that we had, like, a 50/50 chance because basically I thought if- if they play in a totally normal style, I think we'll squeak out a win. But there's always a chance that they can find some weakness in the bot.

Lex Fridman 52:49 Mm-hmm.

Noam Brown 52:50 And if they do... And we're- we're playing, like, for 20 days, 120,000 hands of poker. They have a lot of time to find weaknesses in the system. And if they do, we're gonna get crushed, and that's actually what happened in the previous competition. Um, the humans, you know, they started out. It wasn't like, uh, they were winning from the start, but then they found these weaknesses that they could take advantage of, and for the next, you know, like, 10 days, they were just- just crushing the bot, stealing money from it.

Lex Fridman 53:13 What were the weaknesses they found? Like, maybe overbetting was effective, that kind of stuff? So certain betting strategies worked?

Noam Brown 53:19 What they found is, yeah, overbetting, like betting certain amounts, the bot would have a lot of trouble dealing with those sizes.

Lex Fridman 53:25 Yeah.

Noam Brown 53:25 And then also, um, when it... the bot got into really difficult all-in situations, it- it wasn't able to... because it wasn't doing search, it had to clump different hands together, and it wouldn't... it would treat them identically.

Lex Fridman 53:40 Yeah.

Noam Brown 53:40 Um, and so it wouldn't be able to distinguish, you know, like having a king high flush versus an ace high flush. And in some situations, that really matters a lot. And so, they could put the bot into those situations, and then the bot would- would just bleed money.

Lex Fridman 53:53 Clever humans.

Noam Brown 53:53 Yeah.

Lex Fridman 53:54 Okay. So I didn't realize it was o- over 20 days. So, um, what were the humans like over those 20 days? And what was the bot like?

Noam Brown 54:04 So, we had set up the competition. You know, like I said, there was $200,000 in prize money, and they would get paid a fraction of that depending on how well they did relative to each other.

Lex Fridman 54:14 Yeah.

Noam Brown 54:14 So I was kind of hoping that they wouldn't work together to try to find weaknesses in the bot, but they enter the competition with their, like, number one objective being to beat the bot, and they didn't care about, like, individual glory. They were like, "We're all gonna work as a team to try to take down this bot."

Lex Fridman 54:27 Yeah.

Noam Brown 54:28 And, um... so they immediately started comparing notes. What they would do is they would coordinate looking at different parts of the strategy to try to- try to, you know, find out weaknesses. Um, and then at the end of the day, we actually sent them a log of all the hands that were played and what cards the bot had on each of those hands.

Lex Fridman 54:47 Oh, wow.

Noam Brown 54:47 Yeah. Yeah.

Lex Fridman 54:48 That's- that's gutsy.

Noam Brown 54:49 Yeah. It was honestly a... and I'm not sure why we did that in retrospect, but, um, I mean, I'm glad we did it 'cause we ended up winning anyway, but that... if- if you've ever played poker before, like, that is golden information.

Lex Fridman 54:58 Yeah.

Noam Brown 54:58 I mean, to know... usually when you play poker, you see about a third of the hands to showdown.

Lex Fridman 55:03 Okay

Noam Brown 55:03 Um, and to just hand them all the cards that the bot had on every single hand, that was, um, just- just a gold mine for them.

Lex Fridman 55:11 Yeah, yeah.

Noam Brown 55:11 And so then they would review the hands and try to see, like, okay, could they find patterns in the bot, weaknesses, and could they then- then they would coordinate and study together and try to figure out, "Okay, now this person's gonna explore this part of the strategy for weaknesses. This person's gonna explore this part of the strategy for weaknesses."

Lex Fridman 55:27 (laughs) It's a kind of psychological warfare, showing them the hands.

Noam Brown 55:31 Yeah.

Lex Fridman 55:32 Um, I mean, I'm sure you didn't think of it that way, but, like, doing that means you're confident in the- the bot's ability to win. (laughs)

Noam Brown 55:38 Well, that's- that's one way of putting it. I wasn't, uh, super confident.

Lex Fridman 55:41 Yeah.

Noam Brown 55:42 So, you know, going in, like I said, I think I had, like, 50/50 odds on us winning. The- when we actually- when we announced the competition, the poker community decided to gamble on who would win.

Lex Fridman 55:52 Mm-hmm.

Noam Brown 55:52 And their initial odds against us were like four to one. They- they were really convinced that the humans were gonna pull out a win.

Lex Fridman 55:58 Hmm.

Noam Brown 55:59 Um, the bot ended up winning for three days straight. And even then after three days, the betting odds were still just 50/50. Um, and then at that point, it started to look like the humans were coming back. Um, they started to, like, you know... But- but poker's a very high variance game. Um, and I think what happened is, like, they thought that they spotted some weaknesses that weren't actually there, and then around day eight, it was just very clear that they were getting absolutely crushed. Um, and- and from that point for, I mean, for- for a while there I was super stressed out thinking like, "Oh my God, the humans are coming back and we're just, they've found weaknesses and now we're just gonna lose the whole thing." But no, it- it ended up going in the other direction and the bot ended up, like, crushing them in the long run.

Lex Fridman 56:42 How did it, uh, feel at the end? Like, as a human being, what did, as a person who loves, appreciates the beauty of the game of poker and as a person who appreciates the beauty of AI, did- is there... Did you feel a certain kind of way about it?

Noam Brown 56:59 Uh, I felt a lot of- a lot of things, man. Um, I mean, at that point in my life, I had spent five years working on this project and, um, it was a huge sense of accomplishment. I mean, to spend five years working on something and finally see it succeed. Um, yeah, I wouldn't trade that for anything in the world.

Lex Fridman 57:16 Yeah, because it's a, that's a real benchmark. It's not like, uh, getting a s- some percent accuracy on a dataset. This is like real, this is real world.

Noam Brown 57:26 Yeah.

Lex Fridman 57:26 It's- it's just a game, but it's also a game that means a lot to a lot of people. And this is humans doing their best to beat the machine. So this is a real benchmark, unlike anything else.

Noam Brown 57:36 Yeah. And I mean, this is- this is what I had been dreaming about since I was like 16 playing poker, you know, with my friends in high school. The idea that you could find a strategy, um, uh, you know, approximate the Nash equilibrium, be able to beat all the poker players in the world with it. You know, so to actually see that come to fruition and- and be realized, uh, that was... It's kind of magical.

Lex Fridman 57:58 Yeah, especially when money is on the line too. It's a diff- it's different than chess in that aspect, like, people get... That's why you want to look at betting markets if you want to actually understand what people really think. In the same sense, poker, it's really high stakes because it's money and to solve that game, that's- that's an amazing accomplishment.

Lex Fridman 58:18 So, uh, the leap from that to multi-way six player poker, what's, how difficult is that jump? And what are some interesting differences between heads-up poker and- and multi-way poker?

Noam Brown 58:32 Yeah, so I mentioned, you know, Nash equilibrium in two player zero-sum games. If you play that strategy, you are guaranteed to not lose an expectation no matter what your opponent does. Now, once you go to six player poker, you're no longer playing a two player zero-sum game. And so there was a lot of debate among the academic community and among the poker community about how well these techniques would extend beyond just two player heads-up poker. Now, what I had come to realize is that, um, the techniques actually I thought really would extend to six player poker, because even though in theory they don't give you these guarantees outside of two player zero-sum games, in practice, it still gives you a really strong strategy. Now, there were a lot of complications that would come up with six player poker besides like the game theoretic aspect. I mean, for one, the game is just exponentially larger. Um, so the main thing that allowed us to go from two player to six player was the idea of depth-limited search. So I said before, like, you know, we would do search, we would plan out, the bot would plan out, like, what- what it's going to do next and for the next several moves. And in Libratus, that search was done extending all the way to the end of the game. So it would have to start, um... It- it from- from the turn onwards, like looking maybe 10 moves ahead, um, it would have to figure out what it was doing for all those moves. Now when you get to six player poker, it can't do that exhaustive search anymore because the game is just way too large. Um, but by only having to look a few moves ahead and then stopping there and substituting a value estimate of, like, how good is that strategy at that point, then we're able to do a much more scalable form of search.

Lex Fridman 1:00:14 Is there something cool? We're looking at the paper right now. Is there something cool in the paper in terms of graphics? A game tree traversal via Monte Carlo.

Noam Brown 1:00:22 I think if you go down a bit, uh...

Lex Fridman 1:00:25 Uh, figure one, an example of equilibrium selection problem. Ooh, so yeah, uh, what do we know about equilibria when is- there's multiple players?

Noam Brown 1:00:34 So when you go outside of two player zero-sum, so a Nash equilibrium is a set of strategies, like one strategy for each player, where no player has an incentive to switch to a different strategy. Um, and so you can kind of think of it as like, imagine you have a game where there's a- a ring, um, that's actually the visual here. You got a ring and the object of the game is to be as far away from the other players as possible. There's... A- a Nash equilibrium is for all the players to be spaced equally apart around this ring, but there's infinitely many different Nash equilibria, right? There's infinitely many ways to space four dots along a ring.And if every single player independently computes a Nash equilibrium, then there's no guarantee that the joint strategy that they're all playing, um, is going to resu- is going to be a Nash equilibrium. There- they're just gonna be like random dots scattered along this ring rather than four coordinated dots being equally spaced apart.

Lex Fridman 1:01:28 Is it possible to sort of optimally do this kind of selection, to do the selection abo- um, of the equilibria you're chasing? So, is there like a meta problem to be solved here?

Noam Brown 1:01:40 So the meta problem is in some sense, um, how do you- how do you understand the Nash equilibrium that the other players are going to play. Um, and- and even if you do that, again, like, there's no guarantee that you're going to win. So-

Lex Fridman 1:01:54 Yeah.

Noam Brown 1:01:54 ... you know, if you're playing, uh, if you're playing Risk, like I said, and- and all the other players decide to team up against you, you're gonna lose. Nash equilibrium doesn't help you there. And so there was this big debate about whether Nash equilibrium and all these techniques that compute it are even useful once you go outside of two-player zero-sum games. Now, I think for many games there is a valid criticism here and I think when we talk about- when we go to something like Diplomacy, we run into this issue that the approach of a- trying to approximate a Nash equilibrium doesn't really work anymore. But it turns out that in six-player poker, um, because six-player poker is such an adversarial game, um, where none of the players really try to work with each other, the techniques that were used in two-player poker to try to approximate an equilibrium, those still end up working in practice in- in six-player poker as well.

Lex Fridman 1:02:45 The- there's some deep way in which six-player poker is just a bunch of heads-up poker, like, games in one. It's like a- it's like embedded in it. So the competitiveness, um, is more fundamental to poker than the cooperation.

Noam Brown 1:03:01 Right, yeah. Poker is just such an adversarial game, there's no real cooperation. In fact, you're not even allowed to cooperate in poker. It's considered collusion. It's against the rules. Um, and so for that reason, the techniques end up working really well and I think that's true more- more broadly in extremely adversarial games in general.

Lex Fridman 1:03:18 But that's sort of in practice versus being able to prove something.

Noam Brown 1:03:22 That's right. Nobody has a proof that that's the case and it could be that- that six-player poker belongs to some class of games where apro- approximating a Nash equilibrium through self-play provably works well. Um, and, you know, there are other classes of games beyond just two-player zero-sum where this is proven to work well. So there are these, you know, kinds of games called potential games which I won't go into, it's kind of like a complicated concept, but, um, there are classes of games where, uh, this approach to approximating a Nash equilibrium is proven to work well. Now, six-player poker is not known to belong to one of those classes, but it is possible that there is some class of games where it either provably performs well or it provably performs not that badly.

Lex Fridman 1:04:04 So, what are some interesting things about, uh, Pluribus that was able to achieve human-level performance on this, or superhuman-level performance on the six-player version of poker?

Noam Brown 1:04:15 I- personally, I think the most interesting- interesting thing about Pluribus is that it was so much cheaper than Libratus. I mean, Libratus, if you had to put a price tag on- on the computational resources that went into it, I would say the final training run took about $100,000. You go to Pluribus, the final training run would cost, like, less than $150 on AWS.

Lex Fridman 1:04:38 Is this normalized to, uh, computational inflation? So meaning, uh, does- is- is this just- does this just have to do with the fact that Pluribus was trained, like, a year later?

Noam Brown 1:04:49 No, no, no. It's not- it's- I mean, first of all, like, yeah, computing resources are- are getting cheaper every day and, like- but you're not gonna see a- a thousand-fold decrease in the computational resources over two years, um, or even anywhere close to that. The- the real improvement was algorithmic improvements, and in particular, the ability to do depth-limited search.

Lex Fridman 1:05:08 So it- does depth-limited search also work for Libratus?

Noam Brown 1:05:12 Yep. Yes. So, where this depth-limited search came from is, you know, I- I developed this technique and, um, ran it on two-player poker first and that reduced the computational resources needed to make an AI that was superhuman from, you know, $100,000 for Libratus to something you could train on your laptop.

Lex Fridman 1:05:31 What do you learn from that, um, from that discovery?

Noam Brown 1:05:37 What I would take away from that is that algorithmic improvements really do matter.

Lex Fridman 1:05:40 How would you describe the more general case of limited-depth search? So it's basically constraining the scale, temporal or in some other way, of the computation you're doing in some clever way. So, like, with- i- i- like, how else can you significantly constrain computation, right?

Noam Brown 1:05:59 Well, I think the idea is that we want to be able to leverage search as much as possible and the way that we were doing it in Libratus required us to search all the way to the end of the game. Now, if you're playing a game like chess, the idea that you're gonna search always to the end of the game i- is kinda unimaginable, right? Like, there's just so many situations where you just won't be able to use search in that case or the cost would be, um, you know, prohibitive. And this technique allowed us to leverage search and without having to pay such a huge computational cost for it and be able to apply it more broadly.

Lex Fridman 1:06:31 So to what degree did you use neural nets for, uh, Libratus and Pluribus and more generally, what role do neural nets have to play in, um, in superhuman-level performance in poker?

Noam Brown 1:06:44 So we actually did not use neural nets at all for Libratus or Pluribus and a lot of people found this surprising back in 2017, I think they find it surprising today, um, that we were able to do this without using any neural nets. Um, and I think the reason for that, I mean, I think neural nets are, um, incredibly powerful and the techniques that are used today, even for poker AIs, do rely, uh, quite heavily on neural nets. Um, but it wasn't the main challenge for poker. Like, I think what neural nets are really good for-If you're in a situation where finding features for a value function is really difficult, then neural nets are really powerful. And this was the problem in Go, right? Like the problem in Go was that... Or the final problem in Go at least, was that nobody had a good way of looking at a board and figuring out who was winning or loo- and, and describing, um, through a simple algorithm who was winning or losing. And so there, neural nets were super helpful because you could just feed in a ton of different board positions into this neural net and it would be able to predict then who was winning or losing. But in poker, the features weren't the challenge. The f- the challenge was how do you design a scalable algorithm that would allow you to find this balance strategy, um, that would understand that you have to bluff with the right probability?

Lex Fridman 1:08:05 So, can that be somehow incorporated into the, the value function, this, the complexity of poker that you've described?

Noam Brown 1:08:14 Yeah, so the way the value functions work in poker, like the latest and greatest poker AIs, they do use neural nets for the value function. The way it's done is, is very different from how it's done in a game like chess or Go because in poker, you have to reason about beliefs.

Lex Fridman 1:08:31 Right.

Noam Brown 1:08:31 And so the value of a state depends on the beliefs that players have about what the different cards are. Like if you have pocket aces, then whether that's a really, really good hand or just an okay hand depends on whether you know I have pocket aces, right? Like if you know that I have pocket aces, then if I bet, you're gonna fold immediately. But if you think that I have a really bad hand, then I could bet with pocket aces and make a ton of money. So, the value function in poker these days takes the beliefs as an input, which is very different from like how, how chess and Go AIs work.

Lex Fridman 1:09:09 So as a person who appreciates the game, uh, who do you think is the greatest poker player of all time?

Noam Brown 1:09:17 That's a, that's a tough question. Um-

Lex Fridman 1:09:19 Can an AI help answer that question? Can you, can it actually a- analyze the quality of play, right? So the AIs chess engines can, can give estimates of the quality of play, right? Um, I wonder if there's a... Is there an Elo rating type of system for poker? I suppose you could, but there's just not enough. You would have to play a lot of games, right? A very large number of games. Like, more than you would in chess. The deterministic game makes it easier to estimate Elo, I think.

Noam Brown 1:09:50 I think it is much harder to estimate something like Elo rating in poker. I think it's doable. The problem is that the, the game is very high variance. So you could play... You could be profitable in poker for a year and you could actually be a bad player just because the variance is so high. I mean, you've got top professional poker players that would lose for a year just because they're on a really bad, um, bad streak.

Lex Fridman 1:10:12 So yeah, so for Elo, you have to have a nice clean way of saying if player A played player B and A beats B, that says something, that's a signal. In poker, that's a very noisy signal.

Noam Brown 1:10:24 It's a very noisy signal. Now there is a signal there and so you, you could do this, this calculation, it would just be much harder. Um, but the same way that AIs have now taken over chess and, you know, all the top professional chess players train with, with AIs, the same is true for poker. Um, the game has become, uh, a very computational, um... People train with AIs to try to find out where they're making mistakes, um, try to learn from the AIs to improve their strategy. Uh, so now, yeah, so th- the game has been revolutionized es- es- in the past five years by, by the development of AI in this sport.

Lex Fridman 1:11:00 The skill with which you avoided the question of the greatest of all time was impressive.

Noam Brown 1:11:05 So my feeling is that it's a difficult, it's a difficult question because just like in chess where you can't really compare Magnus Carlsen today to Garry Kasparov, um, because the game has evolved so much. Um, the poker players today are so far beyond the, the, the skills of like people that were playing even 10 or 20 years ago. Um, so you look at the, the kinds of like all-stars that were on ESPN at like the height of the poker boom, pretty much all those players are actually not that good at the game today. At at least the, at least the, the strategy aspect. I mean, they might still be good at like reading the player at the other side of the table and trying to figure out like are they bluffing or not, but in terms of the actual like computational strategy of the game, um, a lot of them have really struggled to keep up with that development. Now, so for that reason, I'll, I'll give an answer and I'm gonna say Daniel Negreanu, who you actually had on the podcast recently. I saw it was a great episode.

Lex Fridman 1:12:00 He's gonna love this so much.

Noam Brown 1:12:02 Yeah. (laughs) And-

Lex Fridman 1:12:03 (laughs) And Phil's gonna hate this so much.

Noam Brown 1:12:06 (laughs) A- and I'm, I'm gonna give him, I'm gonna give him credit because he is one of the few like old school really strong players that have kept up with the development of AI.

Lex Fridman 1:12:15 So he is trying to... He's constantly studying the, the game theory optimal way of playing.

Noam Brown 1:12:19 Exactly, yeah. And I think a lot of, a lot of the old school poker players have just kind of given up on that aspect and, and I gotta give Daniel Negreanu credit for, for keeping up with all the developments that are happening in the sport.

Lex Fridman 1:12:31 Yeah, it's fascinating to watch and it's fascinating to watch where it's headed. Um, yeah, so there you go. Some love for Daniel. Quick pause. Bathroom break?

Noam Brown 1:12:40 Yeah, let's do it.

Lex Fridman 1:12:42 Let's go from poker to diplomacy. What is at, at a high level the game of diplomacy?

Noam Brown 1:12:49 Yeah, so I talked a lot about two-player zero sum games and what's interesting about diplomacy is that it's very different from these like adversarial, uh, games like chess, Go, poker, even StarCraft and Dota. Diplomacy has a much bigger cooperative element to it. It's a seven player game. It was actually created in the '50s, um, and it takes place, uh, before World War I. It's like a map of Europe with seven great powers, um, and they're all trying to form alliances with each other. There's a lot of negotiation going on. Um, and so the whole focus of the game is on, uh...... forming alliances with the other players to take on the other players.

Lex Fridman 1:13:30 England, Germany, Russia, Turkey, Austria, Hungary, Italy, and France.

Noam Brown 1:13:35 That's right, yeah. So, the way the game works is on each turn you spend about, you know, five to 15 minutes talking to the other