Jeff Hawkins: The Thousand Brains Theory of Intelligence | Lex Fridman Podcast #208

Lex Fridman 0:00 The following is a conversation with Jeff Hawkins, a neuroscientist seeking to understand the structure, function, and the origin of intelligence in the human brain. He previously wrote the seminal book on the subject, titled On Intelligence, and recently, a new book called A Thousand Brains, which presents a new theory of intelligence that Richard Dawkins, for example, has been raving about, calling the book, quote, "Brilliant and exhilarating." I can't read those two words and not, uh, think of him saying it in his British accent. Quick mention of our sponsors: Codecademy, Bioptimizers, ExpressVPN, Eight Sleep, and Blinkist. Check them out in the description to support this podcast. As a side note, let me say that one small but powerful idea that Jeff Hawkins mentions in his new book is that if human civilization were to destroy itself, all of knowledge, all our creations would go with us. He proposes that we should think about how to save that knowledge in a way that long outlives us, whether that's on earth, in orbit around earth, or in deep space, and then to send messages that advertise this backup of human knowledge to other intelligent alien civilizations. The main message of this advertisement is not that we are here, but that we were once here. This little difference somehow was deeply humbling to me, that we may, with some non-zero likelihood, destroy ourselves and that an alien civilization, thousands or millions of years from now, may come across this knowledge store, and they would only, with some low probability, even notice it, not to mention be able to interpret it. And the deeper question here for me is what information in all of human knowledge is even essential? Does Wikipedia capture it or not at all? This thought experiment forces me to wonder, what are the things we've accomplished and are hoping to still accomplish that will outlive us? Is it things like complex buildings, bridges, cars, rockets? Is it ideas like science, physics and mathematics? Is it music and art? Is it computers, computational systems, or even artificial intelligence systems? I personally can't imagine that, uh, aliens wouldn't already have all of these things, in fact, much more and much better. To me, the only unique thing we may have is consciousness itself and the actual subjective experience of suffering, of happiness, of hatred, of love. If we can record these experiences in the highest resolution directly from the human brain such that aliens would be able to replay them, that is what we should store and send as a message. Not Wikipedia, but the extremes of conscious experiences, the most important of which, of course, is love. This is the Lex Fridman Podcast, and here is my conversation with Jeff Hawkins.

Lex Fridman 3:04 We previously talked over two years ago. Do you think there's still neurons in your brain that, uh, remember that conversation, that, uh, remember me and got excited? Like, there's a Lex neuron in your brain that just, like, finally has a purpose?

Jeff Hawkins 3:18 I do remember our conversation, or I have some memories of it, and I've formed additional memories of you in the meantime. Um, I wouldn't say there's a neuron or a neurons in my brain that know you, but there are synapses in my brain that have formed that reflect my knowledge of you and the model I have of you and the world. And whether they're the exact same synapses were formed two years ago, it's hard to say, 'cause these things come and go all the time. But, um, we know from... one thing to note about brains is that when you think of things, you often erase the memory and rewrite it again, so... Yes, but I have a memory of you, and I have... that's instantiated in synapses. There's a simpler way to think about it, Lex. So you have, we have a, a model of the world in your head, and that model is continually being updated. I updated this morning. You offered me this water and said it was from the refrigerator. I remember these things, and so we... and so the model includes where we live, the places we know, the words, the objects in the world. There's this monstrous model, and it's constantly being updated, and people are just part of that model. So are animals, so are other physical objects, so are events we've done. So, um, it's, it's no special, in my mind, special place for the memories of humans. I mean, obviously, I know, uh, you know, I know a lot about my wife, um, but... and friends, uh, and so on, but it's, it's not like a, a special place for humans are over here. Uh, but we model everything, and we model other people's behaviors too. So if I said yours was a copy of your mind in my mind, it's just because I know how humans... I've learned how humans behave, and, um, and I've learned some things about you. Um, and that's part of my world model.

Lex Fridman 4:57 Well, I just also mean, uh, the collective intelligence of the human species. I wonder if there's something, um, fundamental to the brain that enables that, so modeling other humans with their ideas.

Jeff Hawkins 5:11 Um, you're, you're, you're actually jumping into a lot of big topics. Like, collective intelligence is a separate topic that a lot of people like to talk about. We could talk about that. Uh, but, um... and so that's interesting. Like, you know, we're not just individuals. We live in society and so on. Uh, but from our research point of view, uh, and so again, let's just talk. We studied the neocortex. It's a sheet of neural tissue. It's about 75% of your brain. It runs on this very repetitive algorithm. It's a very repetitive circuit. And so you can apply that algorithm to lots of different problems, but it's all... underneath, it's the same thing. We're just building this model. So from our point of view, we wouldn't look for these special circuits someplace buried in your brain that might be related to other, you know, understanding other humans. It's more like, you know, how do we build a model of anything? How do we understand anything in the world? And humans are just another part of the things we understand.

Lex Fridman 6:05 So there's nothing, uh, there's nothing to the brain that knows the emergent phenomena of collective intelligence?

Jeff Hawkins 6:11 Well, I certainly know about that. I've heard the terms, I've read-

Lex Fridman 6:14 No, but that's-

Jeff Hawkins 6:15 Right? Right? Well, okay, right.

Lex Fridman 6:16 ... as an idea.

Jeff Hawkins 6:17 Well, I think we have language which is- is sort of built into our brains, and that's a key part of collective intelligence. So there are some, uh, you know, uh, prior assumptions about the world we're gonna live in. When we're born, we're not just a blank slate. Um, and so, you know, did we evolve to take advantage of those situations? Yes. But again, we study only part of the brain, the neocortex. There's other parts of the brain that are very much involved in societal interactions and human emotions and, um, and how we interact and even societal, um, uh, issues about, you know, how we are... how we interact with other people, when we support them, when we're greedy and things like that.

Lex Fridman 6:55 I- I mean, certainly the brain is a great place where to study intelligence. I wonder if it's the fundamental, um, atom of intelligence.

Jeff Hawkins 7:06 Well, I- I would say it's- it's- it's absolutely an essential component. Even if you believe in collective intelligence as, um, hey, that's where it's all happening, that's what we need to study, which I don't believe that, by the way. I think it's really important, but I don't think that is the thing. Um, but even if you do believe that, then you have to understand how the brain works in- in doing that. Um, it's... you know, it's more like we are intelligent in... we are intelligent individuals, and together, we are much more magnified, our intelligence. We can do things which we couldn't do individually. But even as individuals, we're pretty damn smart, and we can model things and understand the world and interact with it. So, um, to me, if you're gonna start someplace, you need to start with the brain. Um, then you could say, well, how do brains interact with each other? And what is the nature of language? And how do we share models, that I've learned something about the world, how do I share it with you? Which is really what, you know, sort of communal intelligence is. I know something, you know something, we've had different experiences in the world. I've learned something about brains, maybe I can impart that to you. You've learned something about, you know, physics and you can impart that to me. Um, but it all comes down to... e- even just the epistemological question of, well, what is knowledge and how do you represent it in the brain? Right? Uh, it's not... that's where it's gonna reside, right? Or in our writings.

Lex Fridman 8:23 It's obvious that, uh, human collaboration, human interaction is how we build societies.

Jeff Hawkins 8:29 Yeah.

Lex Fridman 8:29 Right? But some of the things you talk about and work on, s- some of those elements of what makes up an intelligent entity is there with a single person.

Jeff Hawkins 8:40 Oh, absolutely. I mean, it h- it'd be... we can't deny that the brain is the core element here in- in, uh... at least I ca- I think it's obvious the brain is the core element in all theories of intelligence. Uh, it's where knowledge is represented, it's where knowledge is created. Um, we interact, we share, we build upon each other's work, but, uh, without a brain, you'd have nothing. You know, there would be no intelligence without brains. And so, um, uh, so that's where we start. I got into this field because I just was curious as to who I am. You know, how... you know, how do I think? What's going on in my head when I'm- when I'm thinking? What does it mean to know something? You know, I can ask what it means for me to know something independent of how I learned it from you or from someone else or from society. What does it mean for me to know that I have a model of you in my head? What does it mean to know I know what this microphone does and how it works physically, even though I can't see it right now? Um, how do I know that? Um, what does it mean? How do the neurons do that at the fundamental level of neurons and synapses and so on? Those are really fascinating questions, and, uh, I'm happy to- to... just happy to understand those, if I could.

Lex Fridman 9:44 (laughs)

Lex Fridman 9:46 So in your, um... in your, uh, new book, you talk about our brain, our mind as being made up of many brains. Uh, so the book is called A Thousand Brains: A Thousand Brain Theory of Intelligence. What is the key idea of this book?

Jeff Hawkins 10:02 Uh, the book has three sections, and it has sort of maybe three big ideas. So the first section is all about what we've learned about the neocortex, and that's the thousand brains theory. Just to be complete the picture, the second section is all about AI, and the third section is about the future of humanity. Um, so, um, the thousand brains theory, the- the big idea there, if you've... if I had to summarize into one big idea, is that we think of the- the brain, the neocortex as learning this model of the world, but what we learned is actually there's tens of thousands of independent modeling systems going on. And so each, uh, what we call a column in the cortex, there's about 150,000 of them, is a complete modeling system. So it's a collective intelligence in your head in some sense.

Lex Fridman 10:48 Mm-hmm.

Jeff Hawkins 10:49 So the thousand brains theory says about where do I have knowledge about, you know, this coffee cup, or where's the model of this cell phone? It's not in one place. It's in thousands of separate models that are complementary and they communicate with each other through voting. So this idea that we have, we feel like we're one person, you know, that's our experience, we can explain that, but reality, there's lots of these like... almost like little brains, like, but they're- they're sophisticated modeling systems, about 150,000 of them in each of... a human brain. Um, and that's a total different way of thinking about how the neocortex is structured than, uh, we or anyone else thought of even just five years ago.

Lex Fridman 11:25 So you mentioned you started this journey on just looking in the mirror and trying to understand who you are. So if you have many brains, who are you then?

Jeff Hawkins 11:36 So it's interesting, we have a singular perception, right? You know, we think, oh, I'm just here, I'm looking at you. But it's- it's composed of all these things, like there's sounds, and there's- and there's, um, there's vision, and there's touch, and all kinds of inputs, yet we have the singular perception. And what the thousand brain theory says, we have these models that are visual models, we have o- models that are auditory models, models that are tactile models, and so on. But they vote.

Lex Fridman 11:57 Mm-hmm.

Jeff Hawkins 11:57 And so, um, they send... in the cortex, you can think about the- these columns as about like little grains of rice, 150,000 stacked next to each other. And, um, each one is its own little modeling system, but they have these long range connections that go between them.And we call those voting connections or voting neurons. Um, and so the- the different columns are, you know, try to reach a consensus, like, "What am I looking at? Okay," You know, I, "each one has some ambiguity," but they come to a consensus, "Oh, that's a water bottle I'm looking at." Um, we are only consciously pre- able to perceive the voting.

Lex Fridman 12:31 Mm-hmm.

Jeff Hawkins 12:32 We're not able to perceive anything what goes under- under- under the hood. So the voting is what we're- we're- we're- we're aware of.

Lex Fridman 12:39 The results of the voting.

Jeff Hawkins 12:40 Yeah, the voting. Well, it's- it's- you can imagine it this way. We were just talking about eye movements a moment ago. So, as I'm looking at something, my eyes are moving about three times a second.

Lex Fridman 12:47 Mm-hmm.

Jeff Hawkins 12:48 And with each movement, a completely new input is coming into the brain. It's not repetitive, it's not shifting it around, it's completely new. I'm totally unaware of it. I can't perceive it.

Lex Fridman 12:57 Mm-hmm.

Jeff Hawkins 12:57 But yet, if I looked at the neurons in your brain, they're going on and off, on and off, on and, on off. But the voting neurons are not.

Lex Fridman 13:02 Mm-hmm.

Jeff Hawkins 13:03 The voting neurons are saying, "You know what? We all agree, even though I'm looking at different parts of this, this is a water bottle right now." And that's not changing and it's at some position and- and pose relative to me. So I have this perception of the water bottle about two feet away from me at a certain pose to me. Um, that is not changing. That's the only part I'm aware of. I can't be aware of the fact that the inputs to the, from the eyes are moving and changing, and all this other stuff happening. So, these long-range connections are the part we can be conscious of.

Lex Fridman 13:29 Mm-hmm.

Jeff Hawkins 13:29 The individual activity in each column is, doesn't go anywhere else. It doesn't get shared anywhere else. It doesn't, there's no way to extract it and talk about it, or extract it and even remember it to say, "Oh, yes, I can recall that." Um, so, but these long-range connections are the things that are accessible to language and to our, you know, it's like the hippocampus, our- our- our memories, you know, our n- short-term memory systems and so on. So we're, we're not aware of 95%, or maybe it's even 98% of what's going on in your brain. Um, we're only aware of this sort of stable, somewhat stable, um, voting outcome of all these things that are going on underneath the hood.

Lex Fridman 14:10 So, what would you say is the basic element in the thousand brains theory of intelligence, of intelligence? Like, what's the atom of intelligence when you think about it? Is it the individual brains? And then what is a brain?

Jeff Hawkins 14:24 Well, let's- let's, can we just talk about what intelligence is first?

Lex Fridman 14:28 Yeah.

Jeff Hawkins 14:28 And then, and then we can talk what the elements are. So in my, in my book, intelligence is the ability to learn a model of the world. So to build internal to your head, a model that represents the structure of everything.

Lex Fridman 14:41 Mm-hmm.

Jeff Hawkins 14:42 You know, th- to know that this is a table, and that's a coffee cup, and this is a gooseneck lamp, and all of the sudden, to know these things, I have to have a model of them in my head. I just don't look at them and go, "What is that?" I already have internal representations of these things in my head, and I had to learn them. I wasn't born with any of that knowledge. Uh, you were, you know, we have some lights in the room here. I, you know, that's not part of my evolutionary heritage, right? It's not in my genes.

Lex Fridman 15:03 Mm-hmm.

Jeff Hawkins 15:04 So, um, we have this incredible model, and the model includes not only what things look like and feel like, but where they are relative to each other and how they behave. I've never picked up this water bottle before, but I know that if I took my hand on that blue thing and I turn it, it'll probably make a funny little sound as the little plastic things detach.

Lex Fridman 15:19 Mm-hmm.

Jeff Hawkins 15:20 And then it'll rotate, and it'll rotate a certain way, and it'll come off. How do I know that, right? 'Cause I have this model in my head. So, the essence of intelligence is our ability to learn a model. And the more sophisticated our model is, the smarter we are.

Lex Fridman 15:32 Mm-hmm.

Jeff Hawkins 15:32 Uh, not that there is a single intelligence, 'cause you can know about, you know a lot about things that I don't know, and I know about things you don't know.

Lex Fridman 15:38 Mm-hmm.

Jeff Hawkins 15:39 And we can both be very smart. But we both learned a model of the world through interacting with it. So that is the essence of intelligence. Then we can ask ourselves, what are the mechanisms in the brain that allow us to do that? And what are the mechanisms of learning? Not just the neural mechanisms. What are the general process by how we learn a model? So that was a big insight for us is like, what are the- the, what is the actual things that... How do you learn this stuff?

Lex Fridman 16:01 Mm-hmm.

Jeff Hawkins 16:01 It turns out, you have to learn it through movement. Um, you can't learn it just by... Uh, that's how we learn. We learn through movement. We learn, um, so you build up this model by observing things and touching them and moving them and walking around the world and so on.

Lex Fridman 16:12 So either you move, or the thing moves, you know?

Jeff Hawkins 16:14 Somehow, yeah. You, obviously you can learn things just by reading a book, something like that. But think about it, if I were to say, "Oh, here's a new house. I want you to learn," you know, what do you do? You have to walk, you have to walk from room to room. You have to open the doors, look around, see what's on the left, what's on the right, all the... As you do this, you're building a model in your head.

Lex Fridman 16:31 Mm-hmm.

Jeff Hawkins 16:32 It's just, that's what you're doing. You can't just sit there and say, "I'm gonna grok the house." No. You know? Or, you could, you don't even want to just sit there and read some description of it, right? (laughs)

Lex Fridman 16:40 (laughs) Yeah.

Jeff Hawkins 16:40 You literally physically interact, and the same with like a smartphone. If I wanna learn a new app, I touch it, and I move things around, and I see what happens when I, when I, uh, do things with it. So that's the basic way we learn in the world.

Lex Fridman 16:50 And by the way, when you say model, you mean something that can be used for prediction in the future? So-

Jeff Hawkins 16:56 It's- it's used for prediction and for behavior and planning.

Lex Fridman 17:01 Right.

Jeff Hawkins 17:01 Here's a-

Lex Fridman 17:02 And- and does a pretty good job-

Jeff Hawkins 17:04 Yeah.

Lex Fridman 17:04 ... at doing so.

Jeff Hawkins 17:04 Yeah. Here's the way to think about model, and a lot of people get hung up on this. So, um, you can imagine an architect making a model of a house.

Lex Fridman 17:13 Mm-hmm.

Jeff Hawkins 17:13 Right? So there's a physical model that's small, and why do they do that? Well, we do that because you can imagine what it would look like from different angles.

Lex Fridman 17:20 Mm-hmm.

Jeff Hawkins 17:20 You can say, "Okay, right, look at it from here, look at it from there." And you can also say, "Well, how- how far would it get from the garage, to the, to the swimming pool?" Or something like that, right? You can imagine looking at this, and you can say, "What would be the view from this location?" So we build these physical models to let you imagine the future and imagine the act- behaviors. Now, we can take that same model and put it in a computer. So we now, today, they'll build mout- models of houses in a computer, and they, and they do that using a set of, um, uh - we'll come back to this term in a moment - reference frames. But basically, you assign a- a reference frame for the house, and you assign different things for the house in different locations. And then, the computer can generate an image and say, "Okay, this is what it looks like from this direction." The brain is doing something remarkably similar to this. Surprising. Um, it's using reference frames. It's building these... It's similar to a model in a computer, which has the same benefits of building a physical model. It allows me to say, "What would this thing look like if it was in this orientation? What would likely happen if I pushed this button? I've never pushed this button before." Or, "How would I accomplish something? I wanna-"... I want to, um, uh, convey a new idea I've learned. Uh, how would I do that? I can imagine in my head, well, I could talk about it, I could write a book, I could do some podcasts.

Lex Fridman 18:28 Mm-hmm.

Jeff Hawkins 18:29 I could, um, you know, maybe tell my neighbor, you know, and, and I can imagine the outcomes of all these things before I do any of them. That's what the model lets you do.

Lex Fridman 18:37 Yeah.

Jeff Hawkins 18:37 Let's just plan the future and imagine the, uh, uh, the consequences of our actions. Prediction, you asked about prediction, prediction is not the goal of the model. Prediction is an inherent property of it, and it's how the model corrects itself.

Lex Fridman 18:52 So prediction is fundamental to intelligence.

Jeff Hawkins 18:55 It's fundamental to building a model and the model's intelligent. And pre- let me go back and be very precise about this. Prediction, you can think of prediction two ways. One is like, "Hey, what would happen if I did this?" That's a type of prediction. Um, that's a key part of intelligence. But using predictions like, "Oh, what this, what is this, this water bottle gonna feel like when I pick it up?" You know, and that doesn't seem very intelligent. But the way to think... one way to think about intelligence and prediction is, it's a way for us to learn where our model is wrong.

Lex Fridman 19:23 Mm-hmm.

Jeff Hawkins 19:23 So if I picked up this water bottle and it felt hot, I'd be very surprised. Or if I picked it up and it was very light, it would be very sur- I'd be surprised. Or if I turn this top and it didn't t- I had to turn it the other way, I'd be surprised.

Lex Fridman 19:35 Mm-hmm.

Jeff Hawkins 19:35 And so all those might have a prediction. Like, okay, I'm gonna do it because I'll drink some water. I'm gonna go, okay, I do this, there it is, I feel it opening, right? What if I had to turn it the other way or what if it, it split in two? Then I say, "Oh my gosh, I, I misunderstood this. I didn't have the right model of this thing."

Lex Fridman 19:48 Mm-hmm.

Jeff Hawkins 19:49 My attention would be drawn to, I'll be looking at it going, "Well, how the hell did that happen?"

Lex Fridman 19:52 Yeah.

Jeff Hawkins 19:52 You know, "Why did it open up that way?" And I would update my model by doing it. Just by looking at it and playing around with it, I'd update it and say, "This is a new type of water bottle."

Lex Fridman 20:00 But you... So you're talking about sort of, uh, complicated things like a water bottle.

Jeff Hawkins 20:05 Yeah.

Lex Fridman 20:05 But this also applies for just basic v- vision, just like seeing things. It's almost like a precondition of just perceiving the world is predicting it.

Jeff Hawkins 20:14 Mm-hmm.

Lex Fridman 20:15 It's just everything that you see is first passed through your prediction of what

Jeff Hawkins 20:21 Everything you see and feel. In fact, this, this was the insight I had, uh, back in the late '80s, uh, no, excuse me, early '80s, and, um, and other people have reached the same idea, is that every sensory input you get, not just vision, but touch and hearing, you have an expectation about it-

Lex Fridman 20:39 Yeah.

Jeff Hawkins 20:39 ... and, um, a prediction. Sometimes you can predict very accurately, sometimes you can't, right? I can't predict what next word's gonna come out of your mouth, but as you start talking, I will make better and better predictions. And if you talked about some topics, I'd be very surprised. Um, so I have this sort of background prediction that's going on all the time for all of my senses. Uh, again, I, the way I think about that is this is how we learn. It's, it's more about how we learn. It's a test of our understanding. Our predictions are a test. Do... is this really a water bottle? If it is, I shouldn't see, you know, a little finger sticking out the side.

Lex Fridman 21:13 Mm-hmm.

Jeff Hawkins 21:14 And if I saw a little finger sticking out, I was like, "What the hell is going on?" (laughs) You know, that's not normal. Um, that's w-

Lex Fridman 21:19 I mean, that's fascinating that... just let me linger on this (laughs) for, for a second. I- it, it really honestly feels that prediction is fundamental to everything, uh, to the way our mind operates, to intelligence. So like, it's just a different way to see intelligence, which is like everything starts at prediction.

Jeff Hawkins 21:40 And prediction requires a model. You can't predict something unless you have a model of it.

Lex Fridman 21:45 Right. But the action is prediction. So like, the thing, the thing the model does is prediction and-

Jeff Hawkins 21:51 But it also... yeah, and you but n- but you can then extend it to things like, "Oh, what would happen if I took this today, I went and did this? What would be likely to occur?" Or how... you can extend prediction to like, "Oh, I want to get a promotion at work. Um, what action should I take?" And you can say, "Well, if I did this, I predict what might happen. If I s- spoke to someone, I predict what might happen." So it's not just low-level predictions-

Lex Fridman 22:13 Yeah, it's all predictions.

Jeff Hawkins 22:14 ... it's all predictions.

Lex Fridman 22:15 It's like this black box that you can ask basically any question.

Jeff Hawkins 22:18 Yeah.

Lex Fridman 22:18 Low level or high level.

Jeff Hawkins 22:19 So we started off with that observation that's all... it's, it's just this non-stop prediction, and I write about this in the book about... and then we ask, well, how do neurons actually make predictions physically? Like, what does the neuron do when it makes a prediction? And, um, or the neural tissue does when it makes predictions. And then we ask, what are the mechanisms by how we build a model that allows you to make predictions? So we started with prediction as sort of the fundamental research agenda, if in some sense.

Lex Fridman 22:45 Mm-hmm.

Jeff Hawkins 22:45 Like, and said, "Well, if we understand how the brain makes predictions, we'll understand how it builds these models and how it learns," and that's the core of intelligence. So it was like-

Lex Fridman 22:53 Mm-hmm.

Jeff Hawkins 22:53 ... it was the key that got us in the door to say that is our research agenda, understand predictions.

Lex Fridman 22:59 So in this whole process, where does intelligence originate, would you say? So it w- (laughs) you know, if we look at things that are much less intelligence than humans and you start to build up a human through the process of evolution, where's this magic thing that, uh, has a, a prediction model or a model that's able to predict-

Jeff Hawkins 23:23 Yeah.

Lex Fridman 23:23 ... that starts to look a lot more like intelligence?

Jeff Hawkins 23:25 Yeah.

Lex Fridman 23:25 Is there a place where... Richard Dawkins wrote an introduction to your, uh, to your book, an excellent introduction. I mean, uh, uh, it's, it puts a lot of things into context, and it's funny just looking at parallels for your book and Darwin's Origin of Species. So Darwin wrote about the origin of species, so what is the origin of intelligence?

Jeff Hawkins 23:48 Yeah. Well, we have a theory about it, and it's just that, it's a theory. Theory goes as follows. As soon as living things started to move, they're not just floating in sea, they're not just a plant, you know, grounded someplace. As soon as they started to move, there was an advantage to moving intelligently, to moving in certain ways, and there's some very simple things you can do. You know, bacteria or single-cell organisms can move towards a source of gradient of food or something like that. But an animal that might know where it is and know where it's been and how to get back to that place or an animal that might say, "Oh, there was a source of food someplace. How do I get to it?" Or there was a, a danger. "How do I get...?" Or, "There was a mate, how do I get to them?"... um, there was a big evolutionary advantage to that. So early on, there was a pressure to start understanding your environment, like, "Where am I? And where have I been? And what happened in those different places?" So we still have this neural mechanism in our brains. Um, it's in, in the, in the mammals, it's in the hippocampus and entorhinal cortex. These are older parts of the brain, um, and these are very well-studied. Um, we build a map of the, of our environment. So th- these neurons in these parts of the brain know where I am in this room and where the door was and things like that.

Lex Fridman 25:07 So a lot of other mammals have this (overlapping dialogue)

Jeff Hawkins 25:09 All mammals have this, right?

Lex Fridman 25:10 Okay.

Jeff Hawkins 25:11 And alm- almost any, any animal that knows where it is and get around must have some mapping system, must have some way of saying, "I've learned a map of my environment." I have hummingbirds in my backyard, and they, and they go to the same places all the time, and they have to... They must know where they are. They just know where they are when to f- They're not just randomly flying around. They know. They know particular flowers they come back to. So we all have this, and it turns out it's very tricky to get neurons to do this-

Lex Fridman 25:38 Mm-hmm.

Jeff Hawkins 25:39 ... to build a map of an environment. It's just... And so we now know there's this, these, these famous studies that's still very active about place cells and grid cells and these other types of cells in the older parts of the brain and how they build these maps of the world, and it's really clever. It's obviously been under a lot of evolutionary pressure over a long period of time to get good at this.

Lex Fridman 25:57 Mm-hmm.

Jeff Hawkins 25:57 So animals know, know where they are. What we think has happened, uh, and there's a lot of evidence to suggest this, is that, that mechanism we learn to map, like, a, a space, um, is, was repackaged. The same type of neurons was repackaged into a more compact form, and that became the cortical column, and it was, it was in some sense genericized, if that's a word. It was turned into a very specific thing about learning maps of environments-

Lex Fridman 26:27 Mm-hmm.

Jeff Hawkins 26:28 ... to learning maps of anything, learning a model of anything, not just your space but coffee cups and so on, and it got sort of repackaged into a more compact version, ma- a more universal version, and then replicated.

Lex Fridman 26:43 Mm-hmm.

Jeff Hawkins 26:43 So the reason we're so flexible is we have a very generic version of this mapping algorithm, and we have 150,000 copies of it.

Lex Fridman 26:51 Sounds a lot like the progress of deep learning. (laughs)

Jeff Hawkins 26:54 How so?

Lex Fridman 26:55 Uh, (laughs) so take neural networks that, uh, seem to work well for a specific task, uh, compress them, and, uh, multiply it by a lot, and then you just stack them on top of-

Jeff Hawkins 27:09 Yeah.

Lex Fridman 27:09 It's like the story of transformers in, uh- (laughs)

Jeff Hawkins 27:12 Yeah. But interesting-

Lex Fridman 27:13 ... natural language processing (overlapping dialogue)

Jeff Hawkins 27:13 ... deep learning networks, they end up either replicating an element, but you still need the entire network to do anything.

Lex Fridman 27:20 Right.

Jeff Hawkins 27:20 Here, what, what's going on is each individual element is a complete learning system.

Lex Fridman 27:25 Mm-hmm.

Jeff Hawkins 27:26 This is why I can take a human brain, cut it in half, and it still works.

Lex Fridman 27:29 Mm-hmm.

Jeff Hawkins 27:29 (laughs) You know, it's, it's pretty amazing.

Lex Fridman 27:31 It's fundamentally distributed.

Jeff Hawkins 27:33 It's fundamentally distributed, complete modeling systems. So, um, but that's, that's our story we like to tell. I, I, I would guess it's, it's likely largely right. Um, but, you know, it's, there's a lot of evidence supporting that story, this evolutionary story. Um, the, the thing which brought me to this idea is that the human brain, um, got big very quickly, so that, that led to the proposal a long time ago that, well, there's this common element, just instead of creating new things, it just replicated something. We also are extremely flexible. We can learn things that we had no history about.

Lex Fridman 28:10 Mm-hmm.

Jeff Hawkins 28:10 Right? And so that tells you that the learning algorithm is very, uh, generic. It's very kind of universal 'cause it's, it doesn't assume any p- prior knowledge about what it's learning, and, um, and so you combine those things together, um, and you say, "Okay, well, h- how did that come about? Where did that universal algorithm come from?" It had to come from something that wasn't universal. It came from something that was more specific. And so anyway, this led to our hypothesis that you would find grid cells and place cell equivalents in the neocortex, and when we first published our first papers on this theory, we didn't know of evidence for that. It turns out there was some, but we didn't know about it. Uh, and since then, um... So then, we became aware of evidence for grid cells in c- in parts of the neocortex, and then now there's been new evidence coming out. There's some, uh, uh, uh, interesting papers that came out just January of this year. So our, one of our predictions was if this evolutionary hypothesis is correct, we would see grid cell/place cell equivalents, cells that work like them, through e- ev- every column in the neocortex, and that's starting to be seen.

Lex Fridman 29:08 What does it mean that, uh... Why is it important that they're present throughout?

Jeff Hawkins 29:12 Because it tells us... Well, y- we're asking about the evolutionary origin of intelligence, right?

Lex Fridman 29:15 Right.

Jeff Hawkins 29:16 So our theory is that these columns in the cortex are working on the same principles th- their modeling systems.

Lex Fridman 29:23 Mm-hmm.

Jeff Hawkins 29:24 And it's hard to imagine how neurons do this, and so we said, "Hey, it's really hard to imagine how neurons could learn these models for things." And we can talk about the details of that if you want.

Lex Fridman 29:34 Mm-hmm.

Jeff Hawkins 29:35 But let's, um... But there's this other part of the brain we know that learns models of environments. So could that mechanism that learns to model this room be used to learn to model the, the water bottle?

Lex Fridman 29:45 Mm-hmm.

Jeff Hawkins 29:46 Is it the same mechanism? So we said it's much more likely the brain's using the same mechanism, which case they would have these equivalent cell types.

Lex Fridman 29:53 Mm-hmm.

Jeff Hawkins 29:54 So it's basically, the whole theory is built on the idea that, um, these columns have reference frames, and they're learning these models, and these, these grid cells create these reference frames. So it's, it's basically, um, the major, in some sense, the major predictive, uh, part of this theory is that we will find these equivalent mechanisms in the, each column in the neocortex, which tells us that's, that, yeah, that's what's they're doing. They're learning these sensory motor models o- of the world. Um, so just to... We're pretty confident that would happen, but now we're seeing the evidence for it.

Lex Fridman 30:24 So the evolutionary process, nature does a lot of copy and paste and see what happens?

Jeff Hawkins 30:28 Yeah. Yeah. There's no direction to it, but, but, um, it just found out, like, "Hey, if I...... took this, these elements and, and made more of them, what happens? And let's hook them up to the eyes, and let's hook them up to the ears and, and, um-

Lex Fridman 30:40 And that seems to work pretty well-

Jeff Hawkins 30:42 Yeah. (laughs)

Lex Fridman 30:42 ... like, for us. Again, just to take a quick step back to our, uh, conversation of collective intelligence, do you sometimes see that as just another copy and paste aspect, as, uh, copy and pasting these, uh, brains and s- and humans and making a lot of them and then creating, uh, social structures that then almost operate as a single brain?

Jeff Hawkins 31:06 Uh, I wouldn't have said that, but you said it and it sounded pretty good. (laughs) So... (laughs)

Lex Fridman 31:10 So to you, the brain is fundame- is, uh, is like, um, is its own thing?

Jeff Hawkins 31:15 Yeah.

Lex Fridman 31:16 I- I-

Jeff Hawkins 31:16 I mean, uh, our, our goal is to understand how the neocortex works. A, we can argue how essential that is to understanding the human brain, 'cause it's not the entire human brain. You can on- argue how essential that is to understanding human intelligence. You can argue how essential it is to, um, to, uh, you know, uh, s- sort of communal intelligence. Um, I- I'm, I'm not... I didn't... Our goal was to understand the neocortex.

Lex Fridman 31:38 Yeah. So what is the neocortex and where does it fit in, um, the various aspect of what the brain does? Like, how important is it to you?

Jeff Hawkins 31:46 Well, obviously, again, we... I mentioned it again i- i- in the beginning, it's a, it's about 70% to 75% of the volume of a human brain. So it, you know, it dominates our brain in terms of size. Not in terms of number of neurons, but in terms of size. Uh-

Lex Fridman 32:00 Size is- isn't everything, Jeff.

Jeff Hawkins 32:02 I know. (laughs) But, but it's, it's nothing, it's nothing being... (laughs) It's not that.

Lex Fridman 32:06 (laughs)

Jeff Hawkins 32:06 We know that all high-level vision, hearing, and touch happens in the neocortex. We know that all language occurs and is understood in the neocortex, whether that's spoken language, written language, sign language, whether language of mathematics, language of physics, music, ma- you know. We know that all high level pl- planning and thinking occurs in the neocortex. If I were to say, you know, "What part of your brain designed a computer and understands programming and, and creates music?" It's all the neocortex. So then... But that, that's just like an undeniable fact.

Lex Fridman 32:36 Mm-hmm.

Jeff Hawkins 32:37 Uh, if... But then, there's other parts of our brain are important too, right? Our emotional states, uh, our body, regulating our body. Um, so the way I like to look at it is, you know, could you, can you understand the neocortex without the rest of the brain? And some people say you can't, and I'm, I think absolutely you can. Um, it's not that they're not interacting, but you can understand them. Can you understand the neocortex without understanding the emotions of fear? Yes, you can. You can understand how the system work. It's just a modeling system.

Lex Fridman 33:05 Mm-hmm.

Jeff Hawkins 33:05 I make the analogy in the book that it's, it's like a map of the world. And how that map is used depends on who's using it.

Lex Fridman 33:12 Mm-hmm.

Jeff Hawkins 33:12 So how our map of our world in our neocortex, how we, how we manifest as a human depends on the rest of our brain. What are our motivations? You know, what are my desires? Am I a nice guy or not a nice guy? Am I a cheater or am I a, you know, or not a cheater?

Lex Fridman 33:26 Mm-hmm.

Jeff Hawkins 33:27 Um, uh, you know, how important different things are in my life. So, um, so but the, the... Neocortex can be understood on its own. Um, and, and I say that... As a neuroscientist, I know there's all these interactions, and I'm, I- I don't wanna say I don't know them and that we don't think about them, but-

Lex Fridman 33:44 Mm-hmm.

Jeff Hawkins 33:45 ... from the layperson's point of view, you can say it's a modeling system. Um, I don't generally think too much about the communal aspect of intelligence, which you brought up a number of times already, um, so that's not really been my concern.

Lex Fridman 33:56 I just wonder if there's a continuum.

Lex Fridman 33:59 From the origin of the universe, like, (sighs) this com- pockets of complexities that form-

Jeff Hawkins 34:06 Yeah.

Lex Fridman 34:06 ... living organisms. I wonder if, if we're just... If you look at humans, we feel like we're at the top, but I wonder if there's like just... Where everybody probably... Every living type pocket of complexity is probably... Thinks they're the, uh, pardon the French, they're the shit.

Jeff Hawkins 34:24 Yeah. Well-

Lex Fridman 34:25 They're, they're, they're at the top of the pyramid.

Jeff Hawkins 34:26 Well, if- i- if they're thinking, um...

Lex Fridman 34:29 Well, then- then- then what is thinking? What the...

Jeff Hawkins 34:31 Oh, we can-

Lex Fridman 34:31 In a sense, the, the whole point is, in their sense of the world, they-

Jeff Hawkins 34:37 Yeah.

Lex Fridman 34:38 ... their sense is that they're at- at the top of it.

Jeff Hawkins 34:40 I think, uh-

Lex Fridman 34:41 Like, what does a turtle think?

Jeff Hawkins 34:41 But you're, you're, you're bringing up, you know, the, the, the problems of complexity and complexity theory are, you know, it's a, it's a huge interesting problem in science. Um, and, you know, I think we've made surprisingly little progress in understanding complex systems-

Lex Fridman 34:57 Right.

Jeff Hawkins 34:57 ... in general. Um, and so, you know, the Santa Fe Institute was founded to, to study this. And, and even the scientists there will say, "It's really hard. We haven't really been able to figure out exactly," you know, it... That science hasn't really congealed yet. We, we're still trying to figure out the basic elements of that science. Uh, what... You know, where does complexity come from, and what is it, and how do you define it? Whether it's DNA creating bodies with phenotypes, or it's individuals creating societies, or ants, and you know, markets and so on. It's, it's a very complex thing. I'm not a complexity theorist- person, right? Um, uh, I, I think it's interesting to ask, well, the brain itself is a complex system, so can we understand that? Um, I think we've made a lot of progress understanding how the brain works. So, um, but I haven't s- broadened it out to like, "Oh, well, where are we on the complexity spectrum?" (laughs) You know? It's like-

Lex Fridman 35:47 (laughs)

Jeff Hawkins 35:47 ... um, it's a great question.

Lex Fridman 35:50 I'd prefer for that answer to be, "We're, we're not special." It seems like, if we're honest, most likely, we're not special. So if there is a spectrum-

Jeff Hawkins 36:00 Yeah.

Lex Fridman 36:00 ... we're probably not in some kind of significant place in that spectrum.

Jeff Hawkins 36:03 I, I think there's one thing we could say that we are special, and, and again, only here on Earth, I'm not saying that... Is that if we think about knowledge, what we know, um, we clearly... Human brains have, um... Are the only brains that have certain types of knowledge. We're the only brains on, on this Earth to understand, uh, what the Earth is, how old it is, that the universe is a picture as a whole. We're the only organisms that understand DNA and the origins of, you know, of species. Uh, no other species on, on this planet has that knowledge. So if we think about... I, I like to think about...... you know, one of the endeavors of humanity is to understand the universe as much as we can. Um, I think our species is further along in that, undeniably. Um, whether our theories are right or wrong, we can debate, but at least we have theories. You know, we, we know that, what the sun is and how s- fusion is and how, what black holes are, and, you know, we know general theory of relativity and no other animal has any of this knowledge. (laughs) So, it's in that sense that we're special. Uh, are we special in terms of the, the, the hierarchy of complexity in, in the universe? Probably not.

Lex Fridman 37:16 Can we look at a neuron? Uh, you say that prediction happens in the neuron. What does that mean? So the neuron traditionally is seen as the basic element of the, the brain.

Jeff Hawkins 37:27 So we, I mentioned this earlier, uh, that prediction was our research agenda.

Lex Fridman 37:32 Yeah.

Jeff Hawkins 37:32 We said, "Okay-"

Lex Fridman 37:33 (laughs)

Jeff Hawkins 37:34 "... um, how does the brain make a prediction?" Like, like I, I'm about to grab this water bottle and my brain is predicting what I'm gonna feel on, on, on all my parts of my fingers. If I felt something really odd on any part here, I'd notice it.

Lex Fridman 37:45 Mm-hmm.

Jeff Hawkins 37:45 So my brain is predicting what it's gonna feel as I grab this thing. So what does that... How does that manifest itself in neural tissue, right? We got brains made of neurons and there's chemicals and there's neurons and there's spikes and they're connect- you know, where is, where is the prediction going on?

Lex Fridman 38:01 Mm-hmm.

Jeff Hawkins 38:02 And one argument could be that, well, when I'm predicting something, um, a neuron must be firing in advance. It's like, okay, this neuron represents what you're gonna feel and it's firing, it's sending a spike. And certainly that happens to some extent, but our predictions are so ubiquitous, that we're making so many of them which we're totally unaware of, just the vast majority of them, you have no idea that you're doing this. Um, that it, there wasn't really... We were trying to figure how could this be? Where, where is these, where are these happening, right? And I won't walk you through the whole story unless you insist upon it, but we came to the realization that pred- most of your predictions are occurring inside individual neurons, especially these, the most common neuron, the pyramidal cells. And there are, there's a property of neurons. We, everyone knows or most people know that a neuron is a cell and it has this spike called an action potential and it sends information. But we now know that there's these spikes internal to the neuron, they're called dendritic spikes. They travel along the branches of, of the neuron and they don't leave the neuron. They're just internal only. There's far more dendritic spikes than there are action potentials.

Lex Fridman 39:10 Mm-hmm.

Jeff Hawkins 39:10 Far more. They're happening all the time. And what we came to understand that those dendritic spikes, the ones that are occurring, are actually a form of prediction. They're telling the neuron, the neuron is saying, "I expect that I might become active shortly." And that internal... so the internal spike is a way of saying, "You're gonna... you might be generating an external spike soon. I predicted you're gonna become active." And the, and we've, we've, we've, we wrote a paper in 2016 which explained then how this manifests itself in neural tissue and how it is that this all works together. But the vast ma- we think it's... there's a lot of evidence supporting it. Um, so we, that's where we think that most of these predictions are internal. That's why you can't be pre- they're internal in the neuron, you can't perceive them, but-

Lex Fridman 39:56 Well, uh, f- from understanding the, the prediction mechanism of a single neuron, do you think there's deep insights to be gained about the prediction capabilities of the mini brains within the bigger brain and the brain-

Jeff Hawkins 40:07 Oh, yeah. Yeah, yeah. So having a prediction inside a neur- individual neuron is not that useful, you know? What... so what, right? (laughs)

Lex Fridman 40:14 (laughs)

Jeff Hawkins 40:14 Um, the way it manifests itself in neural tissue is that when a neuron, a neuron emits these spikes, so a very singular type event, if a neuron is predicting that it's gonna be active, it, it emits its spike very, a little bit sooner, just a few milliseconds sooner than it would have otherwise. It's like, I give the analogy in the book, it's like a sprinter on a, on a starting blocks in a, in a race.

Lex Fridman 40:36 Mm-hmm.

Jeff Hawkins 40:37 And if someone says, "Get ready, set," you get up and you're ready to go, and then when they're ready to start, you get a little bit earlier start. So that, it's that, that ready, set is like the prediction and the neuron's like ready to go quicker. And what happens is when you have a whole bunch of neurons together and they're all getting these inputs, the ones that are in the predictive state, the ones that are, are in- anticipating to become active, if they do become active, they, they happen sooner, they disable everything else and it leads to different representations in the brain. So you have to... I- i- it's not isolated just to the neuron. The prediction occurs within the neuron, but the network behavior changes. So what happens under different predictions, different inputs have different representations. So how I, what I predict, um, is gonna be different under different contexts, you know? What my input will be, uh, is different under different contexts. So this is, this is a key to the whole theory, how this works.

Lex Fridman 41:27 So s- for the theory of the thousand brains,

Lex Fridman 41:30 if you were to count the number of brains, how would you do it?

Jeff Hawkins 41:34 The thousand brain theory says that basically every cortical column, uh, in the, in your neocortex is a complete modeling system.

Lex Fridman 41:42 Okay.

Jeff Hawkins 41:42 And that when I ask where do I have a model of something like a coffee cup, it's not in one of those models, it's in thousands of those models. There's thousands of models of coffee cups. That's what the thousand brains theory says.

Lex Fridman 41:52 Then there's a voting mechanism.

Jeff Hawkins 41:53 Then there's a voting mechanism which you leads, which you're, which is the thing you're, which you're conscious of, which leads your singular perception.

Lex Fridman 41:59 Mm-hmm.

Jeff Hawkins 42:00 Um, that's why you, you perceive something. So that's the thousand brains theory. The details of how we got to that theory, um, are complicated. It wasn't that we just thought of it one day. And one of those details where we had to ask, how does a, a model make predictions? And when you talked about just these predictive neurons.

Lex Fridman 42:18 Mm-hmm.

Jeff Hawkins 42:18 That's part of this theory. It's like saying, oh, it's a detail, but we... it was like a crack in the door. It's like, how are we gonna figure out how these neurons build... do this, you know? What is going on here? So we, we just looked at prediction as like, well, we know that's ubiquitous. We know that every part of the cortex is making predictions. Therefore, whatever the predictive system is, it's gonna be everywhere.

Lex Fridman 42:36 Mm-hmm.

Jeff Hawkins 42:37 We know there's a gazillion predictions happening at once. So let's see if we can start teasing apart, y- you know, ask questions about...... you know, how could neurons be making these predictions? And that sort of built up to now what we have this Thousand Brains theory, which is comple-, you know, which is as sim- I can state it simply, but we just didn't think of it. We had to get there step by step, very com- It took years, uh, to get there.

Lex Fridman 42:59 And where does, uh, reference frames fit in? So, yeah.

Jeff Hawkins 43:04 Okay. So again, a reference frame, I, I mentioned, um, earlier about the, you know, a model of a house and I said, "If you're gonna build a model of a house in the computer, you have a reference frame." And you can think of reference frame like, uh, Cartesian coordinates, like X, Y, and Z axes. So I could say, "Oh, I, I'm gonna design a house." I can say, "Well, the, the front door is at this location X, Y, Z and the, the roof is at this location X, Y, Z," and so on. That's a type of reference frame. So turns out for you to make a prediction, and I walk you through the thought experiment in the book where I was predicting what my finger was gonna feel when I touch the coffee cup. It was a ceramic coffee cup, but this one will do. Um, and what I realized is that to make a prediction what my finger was gonna feel like, it's gonna feel different than this, which will feel different if I touch the hole or this thing on the bottom. Make that prediction, the cortex needs to know where the finger is, the tip of the finger-

Lex Fridman 43:55 Mm-hmm.

Jeff Hawkins 43:56 ... relative to the coffee cup, and exactly relative to the coffee cup. And to do that, I have to have a reference frame for the coffee cup. It has to have a way of representing the location of my finger to the coffee cup. And then we realized, of course, every part of your skin has to have a reference frame relative to things it touch, and then we did the same thing with vision. But, so the idea that a reference frame is necessary to make a prediction when you're touching something or when you're seeing something and you're moving your eyes or you're moving your fingers. It's just requirement-

Lex Fridman 44:23 Mm-hmm.

Jeff Hawkins 44:23 ... to, to know what to predict. If I'm have a str- if I have a structure, I'm gonna make a prediction, I have to, uh, I have to know where it is I'm looking or touching it. So then we say, "Well, how do neurons make reference frames?" It's not obvious. Y- you know, X, Y, Z coordinates don't exist in the brain. It's just not the way it works. So that's when we looked at the older part of the brain, the hippocampus and the entorhinal cortex where we knew that in that part of the brain, there's a reference frame for a room or reference frame for environment. Remember I talked earlier about how you could know, make a map of this room.

Lex Fridman 44:53 Mm-hmm.

Jeff Hawkins 44:54 So we said, "Oh, um, that, they are implementing reference frames there." So we knew that reference frames needed to exist in every cortical column and so that was a deductive thing. We just deduced it, has to exist.

Lex Fridman 45:08 So (laughs) you take the old mammalian ability to know where you are in a particular space and you start applying that to higher and higher levels, yeah.

Jeff Hawkins 45:18 Yeah. You, first you apply it to phys-, like where your finger is. So here's the way to think about it. The old part of the brain says, "Where's my body in this room?"

Lex Fridman 45:25 Yeah.

Jeff Hawkins 45:25 The new part of the brain says, "Where's my finger relative to this, this object?"

Lex Fridman 45:30 Yeah.

Jeff Hawkins 45:31 Where is a, the, a, a section of my retina relative to this object? Like where, where is it? I'm looking at one little corner, where is that relative to this patch of my retina?

Lex Fridman 45:39 Yeah.

Jeff Hawkins 45:40 Um, and then we take the same thing and apply it to concepts, mathematics, physics, you know, humanity, whatever you want to think about.

Lex Fridman 45:48 And eventually you're pondering your own mortality.

Jeff Hawkins 45:50 Well, whatever.

Lex Fridman 45:51 (laughs)

Jeff Hawkins 45:51 But the point is, when we think about the world, when we have knowledge about the world-

Lex Fridman 45:56 Yeah.

Jeff Hawkins 45:56 ... how is that knowledge organized, Lex?

Lex Fridman 45:58 Yeah.

Jeff Hawkins 45:58 Where do you, where is it in your head?

Lex Fridman 45:59 Yeah.

Jeff Hawkins 46:00 The answer is, it's in reference frames. So the way I learn the structure of this water bottle, where the features are relative to each other, when I think about history or democracy or mathematics, the same basic underlying structure is happening. There's reference frames for where the knowledge that you're assigning things to. So in the book, I go through examples like mathematics and language and politics. Um, but the, the evidence is very clear in the neuroscience. The same mechanism that we use to model this coffee cup we're gonna use to model high level thoughts. Your, your, your, the demise of the humanity, whatever you want to think about. (laughs)

Lex Fridman 46:34 It's interesting to think about how different are the representations of those higher dimensional concepts, uh, higher level concepts, how different the representation there is in terms of reference frames versus spatial.

Jeff Hawkins 46:48 But the interesting thing, it's, it's, it's a different application, but it's the exact same mechanism.

Lex Fridman 46:54 But isn't there some aspect to, uh, higher level concepts th- they seem to be hierarchical? Like, they just seem to integrate a lot of information into that.

Jeff Hawkins 47:03 So is our physical objects. So take this water bottle, uh, I'm not particular to this brand, but this is a Fiji water bottle-

Lex Fridman 47:12 Mm-hmm.

Jeff Hawkins 47:13 ... and it has, um, a logo on. I use this example in the, in my book, our, our, the, our company's coffee cup has a logo on it. But this object is hierarchical. It is, uh, it's got like a cylinder and a cap, but then has this logo on it and the logo has a word, the word has letters, the letters have different features. And so I don't have to remember, I don't have to think about this. I say, "Oh, there's a Fiji logo on this water bottle."

Lex Fridman 47:36 Yeah.

Jeff Hawkins 47:36 I don't have to go through and say, "Oh, what is the Fiji logo?" It's the F and I and J and I, and there's a hibiscus flower and, and, uh, oh, it has a pes- you know, the stamen on it. I don't have to do that. I just incorporate all of that in some sort of hierarchical representation. I say, um, "You know, put this logo on this water bottle."

Lex Fridman 47:52 Yeah.

Jeff Hawkins 47:53 And, and, and then the logo has a word and the word has letters, all hierarchical.

Lex Fridman 47:57 It's all that stuff is big. It's amazing that the brain instantly just does all that.

Jeff Hawkins 48:01 Yeah.

Lex Fridman 48:01 The idea that there's, there's water, it's liquid, and the idea that you can, uh, drink it when you're thirsty. The idea that there's brands.

Jeff Hawkins 48:09 Yeah. (laughs)

Lex Fridman 48:10 And then there's like all of that information-

Jeff Hawkins 48:12 Yeah.

Lex Fridman 48:12 ... is instantly avai- like-

Jeff Hawkins 48:14 Yeah.

Lex Fridman 48:14 ... built into the whole thing when, once you perceive it.

Jeff Hawkins 48:16 So I wanted to get back to your point about hierarchical representation. Uh, the world itself is hierarchical, right? And I can take this microphone in front of me. I know inside, there's gonna be some electronics. I know there's gonna be some wires, and I know there's gonna be a little diaphragm that moves back and forth. Um, I don't see that, but I know it. Um, so everything in the world is hierarchical. You just go into a room, it's composed of other components, of kitchen has a refrigerator, you know, the, the refrigerator has a door, the door has a hinge, the hinge has screws and pin, yeah, I mean... So anyway, the, the, the modeling system that exists in every cortical column learns the hierarchical structure of objects.So it's a very sophisticated modeling system in this grain of rice. It's hard to imagine, but this grain of rice can do really sophisticated things.

Lex Fridman 48:57 Mm-hmm.

Jeff Hawkins 48:57 It's got 100,000 neurons in it. Um, it's very sophisticated. So the same mechanism that can model a, a water bottle or a coffee cup can model conceptual objects as well. It's, it's, that's the beauty of this discovery that this guy, Vernon Mountcastle, made many, many years ago, which is that there's a, there's a single cortical algorithm underlying everything we're doing. Um-

Lex Fridman 49:21 So, so common sense concepts and higher level concepts are all represented in the same way?

Jeff Hawkins 49:26 They're set in the same mechanisms. Yeah. It's a little bit like, uh, computers, right? All computers are universal Turing machines. Even the little teeny one that's, you know, my toaster, and the big one that's, you know, running some cloud server someplace. Um, they're all running on the same principle, but they can apply to different things. So the brain is all built on the same principle, it's all about learning these models, structured models using movement and reference frames, and it can be applied to something as simple as a water bottle and a coffee cup, and it can be, like, just thinking, like, what's the future of humanity and, you know, (laughs) why do you have a, a hedgehog on your, on your desk?

Lex Fridman 50:02 (laughs)

Jeff Hawkins 50:02 I don't know.

Lex Fridman 50:03 Nobody knows. Well, the...

Jeff Hawkins 50:04 (laughs)

Lex Fridman 50:05 Uh...

Jeff Hawkins 50:05 I think it's a hedgehog. I don't know.

Lex Fridman 50:06 It's, that's right, it's a hedgehog in, in the fog. It's a Russian

Lex Fridman 50:10 reference. Does it give you any inclination or hope about how difficult it is to engineer common sense reasoning? So how complicated this, is this whole process? So looking at the brain, is this a marvel of engineering or is it pretty dumb stuff stacked on top of each other over, uh (laughs) -

Jeff Hawkins 50:31 It can be both. Can't-

Lex Fridman 50:33 ... a pretty extensive copy?

Jeff Hawkins 50:33 It can't be both? It can't, it can't be both, right?

Lex Fridman 50:35 I don't know if it can be bo- uh, both, because, uh, if it's an incredible engineering job, that means it's ve- so evolution did a lot of work. It, uh...

Jeff Hawkins 50:46 Yeah, but then, but then it just copied that, right? So as I said earlier, the figuring out how to model something, like a space, is really hard, and evolution had to go through a lot of trick... And these, these, these cells I was talking about, these grid cells and place cells, they're really complicated. This is not simple stuff, the- this neural tissue works on these really unexpected, weird mechanisms. Um, but it did it, it figured it out. But, but now you could just make lots of copies of it, right? (laughs)

Lex Fridman 51:13 But it, but then finding, yeah, so it's a, it's a very interesting idea that it's a lot of copies of a basic mini brain, but the question is how difficult it is to find that mini brain that you can copy and paste, uh, effectively, like it...

Jeff Hawkins 51:29 Well, we, today, we know enough to build this. I'm sitting here with the, you know, I, I know the steps we have to go. There's still some engineering problems to solve, but we know enough. And this is not like, "Oh, this is an interesting idea. We have to go think about it for another few decades." No, we actually understand it pretty well details. So not all the details, but most of them. So it's complicated, but it is an engineering problem. So in my company, we are working on that. We are basically laid out a roadmap of how we do this. Um, it's not gonna take decades, it's matter of a few years, um, optimistically, but I think that's possible. Um, it's, you know, complex things, if you understand them, you can build them.

Lex Fridman 52:12 So in which domain do you think it's best to build them? Are we talking about robotics, like, uh, entities that operate in the physical world that are able to interact with that world? Are we talking about entities that operate in the digital world? Are we talking about something more like, uh, more specific, like, is done in the, uh, machine learning community where you look at natural language or computer vision? Wh- what, where do you think is easiest to ?

Jeff Hawkins 52:41 It's the first, it's the first two more than the third one, I would say. Um, ag- again, let's just use computers as an analogy. Um, the pioneers in computing, people like John von Neumann and Alan Turing, they created this thing, you know, we now call the universal Turing machine, which is a computer, right? Did they know how it was gonna be applied? Where it was gonna be used? You know, could they inv- envision any of the future? No. They just said, "This is, like, a really interesting computational idea about algorithms and how you can implement them in, in a machine." And we're doing something similar to that today. Like, we are, we are building this sort of universal learning principle that can be applied to many, many different things.

Lex Fridman 53:24 But the, the robotics piece of that-

Jeff Hawkins 53:26 Okay.

Lex Fridman 53:27 ... th- the interactive elements.

Jeff Hawkins 53:27 Okay, all right, let's, let us be specific. You can think of this cortical column as a s- what we call a sensory motor learning system. It has the idea that there's a sensor and that it's moving. That sensor can be physical, it could be, like, my finger and it's moving in the world, it could be, like, my eye and it's physically moving. It can also be virtual. So it could be, um, an example would be, I could have a, uh, a system that lives in the internet that, that actually samples information on the internet and moves by following links. That's-

Lex Fridman 53:56 Mm-hmm.

Jeff Hawkins 53:56 ... that's a sensory motor system, so-

Lex Fridman 53:58 (laughs) something that echoes the, the process of a finger moving along a co- a coffee cup.

Jeff Hawkins 54:03 But in a very, very loose sense. It's, it's, like, again, learning is inherently about discovering the structure in the world, and to discover the structure in the world, you have to move through the world.

Lex Fridman 54:12 Mm-hmm.

Jeff Hawkins 54:13 Even if it's a virtual world, even if it's a conceptual world, you have to move through it. You don't, it doesn't exist in one... It has some structure to it.

Lex Fridman 54:21 Mm-hmm.

Jeff Hawkins 54:21 So here's, here's a couple of predictions, again, what you're talking about. In humans, the same algorithm is, does robotics, right? It moves my arms, my eyes, my body, right? Um, and so in my- in the future, to me, robotics and AI will merge. They're not gonna be separate fields because they're gonna, the- the- the, the algorithms for really controlling robots are gonna be the same algorithms we have in our bread- uh, brain at these sensory motor algorithms. I, today, we're not there, but I think that's gonna happen. And, um...And then so- but not all AI systems will have, be robotics. Um, you can have systems that, uh, have very different types of embodiments. Some will have physical movements, some will have, not have physical movements. It's a very generic learning system. Again, it's like computers, uh, the Turing machine is, is like, doesn't say how it's supposed to be implemented, doesn't tell you how big it is, doesn't tell you what you can apply it to, but it's an int- it's a computational principle. Cortical column equivalent is a computational principle about learning.

Lex Fridman 55:20 Mm-hmm.

Jeff Hawkins 55:20 It's about how you learn and it can be applied to a gazillion things. This is what I think, this is... I think this impact of AI is gonna be as large if not larger than computing has been in the last century, by far. Because it's a f- it's getting at a fundamental thing. It's not a vision system or a learning system, it's a, a, a... It's not a vision system or a hearing system, it is a learning system. It's a fundamental principle how you learn the structure in the world, how you gain knowledge and be intelligent, and that's what The Thousand Brains says was going on. And we have a particular implementation in our head, but it doesn't have to be like that at all.

Lex Fridman 55:51 Do you think there's going to be some kind of impact... Okay, let me ask it another way. What do, uh, increasingly intelligent AI systems do with us humans-

Jeff Hawkins 56:02 (laughs)

Lex Fridman 56:02 ... in the following way? Like, how hard is the human in the loop problem? How hard is it in, to, in, to, to interact the finger on the coffee cup equivalent of having a conversation with a human being? So, how hard is it to fit into our little human world?

Jeff Hawkins 56:19 Uh, I don't... I think it's a lot of engineering problems. I don't think it's a fundamental problem.

Lex Fridman 56:25 Okay.

Jeff Hawkins 56:25 I could ask you the same question, how hard is it for computers to fit into a human world?

Lex Fridman 56:29 Right. That, I mean that's essentially what I'm asking. Like, how, um, much are we, uh, elitist, are we as humans? Like, we try to keep out, uh, systems.

Jeff Hawkins 56:41 I don't know. I, I, I don't know. So I think the... I'm not sure that's the right question. Let's, let's look at computers as an analogy.

Lex Fridman 56:49 Mm-hmm.

Jeff Hawkins 56:49 Computers are a million times faster than us, they do things we can't understand, most people have no idea what's going on when they use computers, right? (laughs)

Lex Fridman 56:56 Mm-hmm.

Jeff Hawkins 56:56 How they, how do we integrate them in our society? Um, well, they're, th- we don't think of 'em as their own entity and they're, they're not living things. Um, we don't afford them rights. Um, we, uh, we rely on them. Our survival a- as a 7 billion people or something like that is relying on computers now. Um-

Lex Fridman 57:16 Don't you think that's a fundamental problem that we s- see them as something we can't, we don't give rights to? So like-

Jeff Hawkins 57:22 Computers?

Lex Fridman 57:23 So yeah, computers. So, uh, robots, computers, intelligent systems, it feels like for them to operate successfully, they would need to have a lot of the elements that we would start having to think about, like, should this entity have rights?

Jeff Hawkins 57:40 I, I don't think so. I, I think it's tempting to think that way. First of all, I don't think anyone, hardly anyone thinks that for computers today. No one says, "Oh, this thing needs a right, I shouldn't be able to turn it off," or, you know, "If I throw it in the trash can, you know, and hit it with a sledgehammer, am I, am I forming a criminal act?" No, no one thinks that.

Lex Fridman 57:59 Mm-hmm.

Jeff Hawkins 57:59 Um, and now we think about intelligent machines, which is where you're going.

Lex Fridman 58:02 Mm-hmm.

Jeff Hawkins 58:03 Um, and, and all, all of a sudden we're like, "Well, now we can't do that." I think the basic problem we have here is that people think intelligent machines will be like us. They're gonna have the same emotions as we do, the same feelings as we do. What if I can build an intelligent machine that ab- absolutely could care less about whether it was on or off or destroyed or not, it just doesn't care, it's just like a map, it's just a modeling system, has no desires to live, nothing.

Lex Fridman 58:28 Is it possible to create a system that can model the world deeply and not care-

Jeff Hawkins 58:35 Yeah.

Lex Fridman 58:35 ... about whether it lives or dies?

Jeff Hawkins 58:36 Absolutely. No question about it.

Lex Fridman 58:38 To me, that's not 100% obvious.

Jeff Hawkins 58:41 It's, it, it's obvious to me. So-

Lex Fridman 58:42 Okay. (laughs)

Jeff Hawkins 58:42 ... we can, we can debate it if we want.

Lex Fridman 58:44 Yeah.

Jeff Hawkins 58:44 Where does your, where does your desire to live come from? It's an old evolutionary-

Lex Fridman 58:50 Yeah.

Jeff Hawkins 58:50 ... design. I mean, we could argue, does it really matter if we live or not? O- objectively no, right?

Lex Fridman 58:57 Right.

Jeff Hawkins 58:57 We're all gonna die eventually.

Lex Fridman 58:58 Yeah.

Jeff Hawkins 58:58 Um, but evolution makes us want to live. Evolution makes us want to fight to live. Evolutionists want to care and love one another and to care for our children and our, and our relatives and our family and, and, and so on. And those are all good things, um, but they come about not because we're smart, because we're animals that grew up. You know? The, the hummingbird in my backyard cares about its offspring, you know? The- i- uh, every living thing in some sense cares about, you know, surviving.

Lex Fridman 59:28 Mm-hmm.

Jeff Hawkins 59:28 But when we talk about creating intelligent machines, we're not creating life, we're not creating evolving creatures, we're not creating living things, we're just creating a machine that can learn really sophisticated stuff. And that machine, it may even be able to talk to us.

Lex Fridman 59:42 Mm-hmm.

Jeff Hawkins 59:42 But it doesn't, it's not gonna have a desire to live unless somehow we put it into that system.

Lex Fridman 59:48 Well, there's learning, right? The, the thing is-

Jeff Hawkins 59:51 But you don't learn to li- want to live, that's built into you. It's-

Lex Fridman 59:54 Well-

Jeff Hawkins 59:55 ... a part of your DNA.

Lex Fridman 59:55 ... so p- people like Ernest Becker argue... So, okay, uh, there's the fact, the finiteness of life. The way we think about it is something we learn, uh, perhaps. So... Okay.

Jeff Hawkins 1:00:08 Yeah.

Lex Fridman 1:00:09 Th-

Jeff Hawkins 1:00:09 And some people decide they don't want to live and some people decide, you know, you can... But the desire to live is built in DNA, right?

Lex Fridman 1:00:15 But I think wha- what I'm trying to get to is, uh, in order to accomplish goals, it's useful to have the urgency of mortality. This is what the Stoics talked about, is meditating in your mortality.

Jeff Hawkins 1:00:26 Yeah.

Lex Fridman 1:00:26 It might be a very useful thing to do, to die and have the urgency of death, and to realize that, to, uh, conceive yourself as an entity that operates in this world that eventually will no longer be a part of this world, and actually conceive of yourself as a conscious entity might be very useful for you to be a system that makes sense of the world. Otherwise, you might get lazy. (laughs)

Jeff Hawkins 1:00:52 Well, okay, we're gonna build these machines, right? So are we talking about building an AIs?

Lex Fridman 1:00:57 What... But we're, we're building the, um...... uh, uh, the, the equivalent of the cortical columns, the, uh-

Jeff Hawkins 1:01:05 The neocortex.

Lex Fridman 1:01:05 ... yeah, the neocortex. And the question is where do they arrive at? 'Cause we're not hard coding everything in. We're, uh-

Jeff Hawkins 1:01:14 Well, well, in terms of, if you build the neocortex equivalent, it will not have any of these desires or emotional states. Now you can argue that that neocortex won't be useful unless I give it some agency, unless I give it some desire, unless I give it some motivation. Otherwise, it'll be as lazy to do nothing. Right? You could argue that. Um, but on its own, it's not gonna do those things. It's just not, it's just not gonna sit there and say, "I understand the world, therefore I care to live." No, it's not gonna say that. It's just gonna say, "I understand the world."

Lex Fridman 1:01:43 Why is that obvious to you? Why, oh, why- why don't-

Jeff Hawkins 1:01:46 Because-

Lex Fridman 1:01:46 Do you think it's pos- okay, let me ask it this way.

Jeff Hawkins 1:01:47 Yeah.

Lex Fridman 1:01:48 Do you think it's possible it will at least, uh, assign to itself, um, agency and, um, perceive itself in this world as being a conscious entity, as a useful way to operate in the world and, and to make sense of the world?

Jeff Hawkins 1:02:07 Well, I think an intelligent machine could be conscious, but that doesn't, does not, again, imply any of these, um, these desires and goals and, and-

Lex Fridman 1:02:14 Mm-hmm.

Jeff Hawkins 1:02:14 ... that you're worried about. It, uh, we can, I have, uh, we can talk about what it means for a machine to be conscious.

Lex Fridman 1:02:20 Mm-hmm.

Jeff Hawkins 1:02:20 Right?

Lex Fridman 1:02:20 And by the way, not worry about, but get excited about. It's not necessarily that we should worry about it.

Jeff Hawkins 1:02:25 Yeah. So, I think-

Lex Fridman 1:02:25 (laughs)

Jeff Hawkins 1:02:25 ... there's a legitimate problem, or not problem, a question to ask. If you build this modeling system, what's it gonna model?

Lex Fridman 1:02:32 Yes.

Jeff Hawkins 1:02:32 Right? What's it, what's its desire?

Lex Fridman 1:02:35 Yeah.

Jeff Hawkins 1:02:35 What's its, what's its goal? What are we applying it to? Right?

Lex Fridman 1:02:38 Mm-hmm.

Jeff Hawkins 1:02:38 So that's an interesting question. Um, one thing if it, and it depends on the application. It's not something that inherent to the modeling system. It's something we apply to the modeling system in a particular way. So if I wanted to make a really smart car, it would have to know about driving and cars and what's important in driving and cars.

Lex Fridman 1:02:58 Mm-hmm.

Jeff Hawkins 1:02:58 It's not gonna figure that out on its own. It's not gonna sit there and say, "You know, I've understood the world and I've decided, you know..." No, no, no. We're gonna have to tell it. We're gonna have to say, like... So I, imagine I make this car really smart. It learns about your driving habits, it learns about the world, and it just, you know... Is it one day gonna wake up and say, "You know what? I'm tired of driving and doing what you want. I think I have better ideas about how to spend my time"?

Lex Fridman 1:03:21 Well, th- there's, okay, I'm not-

Jeff Hawkins 1:03:23 No, it's not gonna do that.

Narrator 1:03:24 It doesn't make sense-

Lex Fridman 1:03:24 Well, part of me is playing a little bit of devil's advocate.

Jeff Hawkins 1:03:26 I can see that.

Lex Fridman 1:03:26 But part of me is also trying to think through this because, uh, I've studied cars quite a bit and I've studied pedestrians and cyclists quite a bit. And there's part of me that thinks that there needs to be, um, more intelligence than we realize in order to drive successfully. Uh, the, that game theory of human interaction seems to require some deep understanding of hu- um, of human nature. That... Okay, when a pedestrian crosses the street, there is some sense, they, they look at a car usually, and then they look away. There's some sense in which they say, "I believe that you're not going to murder me. You don't have the guts to murder me." This is the little dance of pedestrian-

Jeff Hawkins 1:04:16 Yeah.

Lex Fridman 1:04:16 ... car interaction-

Jeff Hawkins 1:04:17 Yeah.

Lex Fridman 1:04:18 ... is saying, "I'm gonna look away and I'm gonna put my life in your hands because I think you're human, you're not gonna kill me." And then the car, in order to successfully operate in, like, Manhattan streets, has to say, "No, no, no. I am going to kill you."

Jeff Hawkins 1:04:34 No.

Lex Fridman 1:04:34 Like a little bit.

Jeff Hawkins 1:04:35 No, nah-

Lex Fridman 1:04:35 There's a little bit of this weird inkling of mutual murder. (laughs)

Jeff Hawkins 1:04:38 What are you talking about? Yeah.

Lex Fridman 1:04:39 And it's a dance.

Jeff Hawkins 1:04:40 Yeah.

Lex Fridman 1:04:40 And we somehow successfully operate through that.

Jeff Hawkins 1:04:42 So do we, do you think you were born with that or did you learn that social interaction?

Lex Fridman 1:04:46 Uh, I think it, it might have a lot of the same elements that you're talking about, which is we're leveraging things we were born with and applying them in the context that, uh, requires learning.

Jeff Hawkins 1:04:58 All right. I would, I would have answered it. I would have said that that kind of interaction is learned 'cause, you know, people in different cultures have different interactions like that. If you cross the street in different cities, in different parts of the world, they have different ways of interacting. I would say that's learned, and I would say an intelligent system could learn that too. But that does not lead... An intelligent system can understand humans. It could understand that w- you know, just like I can study an animal and, and learn something about that animal. You know, I could study apes and learn something about their culture and so on. I don't have to be an ape to know that. Um, I may not be completely, but I can understand something. So an intelligent machine can model that. That's just part of the world. It's just part of the interactions. The question we're trying to get at, will the intelligent machine have its own personal agency that's beyond, you know, what we assign to it, or its, its own personal, you know, goals-

Lex Fridman 1:05:48 Mm-hmm.

Jeff Hawkins 1:05:48 ... or will it evolve and create these things? My confidence comes from understanding the mechanisms w- I'm talking about creating.

Lex Fridman 1:05:55 Mm-hmm.

Jeff Hawkins 1:05:56 This is not hand wavy stuff. It's down to the details we, I, I'm gonna build it and I know what it's gonna look like and I know how it's gonna behave. I know what the kind of things it could do and the kind of things it can't do. Just like when I build a computer, I know it's not gonna, on its own, decide to put another register inside of it. It can't do that. (laughs)

Lex Fridman 1:06:12 Mm-hmm.

Jeff Hawkins 1:06:13 No way. No matter what your software does, it can't add a register to the computer. Um, so in this way when we build AI systems, we have to make choices about the, the, the under- how we embed them.

Lex Fridman 1:06:26 Mm-hmm.

Jeff Hawkins 1:06:26 So I talk about this in the book. I said, you know, it's, a bra- an intelligent system is not just a neocortex equivalent.

Lex Fridman 1:06:31 Mm-hmm.

Jeff Hawkins 1:06:31 You have to have that, but it has to have some kind of embodiment, physical or virtual. It has to have some sort of goals. It has to have some sort of, uh, ideas about dangers, about things that it shouldn't do, like, you know, like, we, we build in safeguards into systems. Uh, we have them in our bodies. We have put them into cars, right? (laughs) You know, my car follows my directions until the day it sees I'm about to hit something and it ignores my directions and puts the brakes on.

Lex Fridman 1:06:56 Mm-hmm.

Jeff Hawkins 1:06:56 So we can build those things in. So that's a very interesting problem, um, how to build those in. I think my, my-... my differing opinion about the risks of AI for most people is that people assume that somehow those things will just appear automatically or it'll evolve, and intelligence itself, uh, begets that stuff, or requires it, but it's not. Intelligence of the neocortex equivalent doesn't require this. The neocortex equivalent just says, "I'm a learning system. Tell me what you want me to learn, and I'll tell you... A- and ask me questions, and I'll tell you the answers." But... And, and that, again, it's again like a map. It doesn't... A map has no intent about things, but you can use it, um, to solve problems.

Lex Fridman 1:07:37 Okay, so the building engineering the neocortex in itself is just creating an intelligent prediction system.

Jeff Hawkins 1:07:46 Modeling system, yes.

Lex Fridman 1:07:46 Model... Oh, sorry. A modeling system?

Jeff Hawkins 1:07:48 Yeah.

Lex Fridman 1:07:49 Uh, that you can use it to then make predictions and then, um... But you can also put it inside a, a thing that's actually acting in this world ?it has...

Jeff Hawkins 1:07:56 Well, that's it.

Lex Fridman 1:07:57 You have to put it inside something. It's... Again, think of the map analogy, right? A map on its own doesn't do anything. Right.

Jeff Hawkins 1:08:03 It's, it's just inert. It's just... It can learn, but it's just inert. So we have to embed it somehow in something to do something.

Lex Fridman 1:08:09 So,

Lex Fridman 1:08:10 so what's your intuition here? You had, you had a (laughs) conversation with Sam Harris recently that was, uh, sort of, um... You've had a bit of a disagreement and you're sticking on this point. You know, Elon Musk, uh, Stuart Russell, kinda have a worry, existential threats of AI. What's your intuition? Why, if we engineer an increasingly intelligent neocortex type of system in the computer, why that shouldn't be a thing that we, we can worry about?

Jeff Hawkins 1:08:41 It was intu... He used the word intuition, and Sam Harris used the word intuition too. And, and when he used that intuition, that word, I immediately stopped and said, "Oh, that's the crux of the problem."

Lex Fridman 1:08:50 Mm-hmm.

Jeff Hawkins 1:08:50 He's using intuition. I'm not speaking about my intuition.

Lex Fridman 1:08:53 Yes.

Jeff Hawkins 1:08:54 I'm speaking about something I understand, something I'm gonna build, something I am building-

Lex Fridman 1:08:57 Mm-hmm.

Jeff Hawkins 1:08:57 ... something I'm, I, I understand completely, or at least well enough to know what... It's not I'm guessing. I know what this thing's gonna do. And I think most people who are worried, they have trouble separating out... I mean, they don't have, they don't have the knowledge or the understanding about like what is intelligence, how does it manifest in the brain? How is it separate from these other functions in the brain? And so they imagine it's gonna be human-like or animal-like. It's gonna have, it's gonna have the same sort of drives and emotions we have. But there's no reason for that. That's just because there's, there's unknown. If you're... If the unknown is like, "Oh my God, you know, I don't know what this is gonna do. We have to be careful. It could be like us, but really smarter."

Lex Fridman 1:09:35 Mm-hmm.

Jeff Hawkins 1:09:35 I'm saying, no, it won't be like us. It'll be really smart, but it won't be like us at all. And, um, and... But I, I'm coming from that not because I'm just guessing, I'm not intuit- uh, u- using intuition. I'm basing it on like, "Okay, I understand how this thing works. This is what it does. L- it makes money to you."

Lex Fridman 1:09:50 Okay. But, uh, to push back... So I, I also disagree with the, the intuitions that Sam has. But, but I also disagree with what you just said, which, you know... What's a good, uh, analogy? So if you look at the Twitter algorithm in, in the early days, just recommender systems, you can understand how recommender systems work. What you can't understand in the early days is when you apply that recommender system at scale to thousands and millions of people, how that can change societies.

Jeff Hawkins 1:10:22 Yeah.

Lex Fridman 1:10:22 So, the, the question is, yes, you're just saying this is how an engineered neocortex works, but the que- like, when you have a very useful, uh, TikTok type of service that get-

Jeff Hawkins 1:10:35 Yup.

Lex Fridman 1:10:35 ... goes viral.

Jeff Hawkins 1:10:36 Yeah.

Lex Fridman 1:10:36 When your neocortex goes viral- (laughs)

Jeff Hawkins 1:10:38 Yeah.

Lex Fridman 1:10:38 ... and then millions of people start using it-

Jeff Hawkins 1:10:40 Yeah.

Lex Fridman 1:10:40 ... can that destroy the world?

Jeff Hawkins 1:10:41 No. Uh, well, first of all, this is back... One thing I wanna say is that, um, AI is a dangerous technology. I don't, I'm not denying that.

Lex Fridman 1:10:48 All technology is dangerous, right?

Jeff Hawkins 1:10:49 Well, an AI may- maybe particularly so.

Lex Fridman 1:10:51 Yeah.

Jeff Hawkins 1:10:52 Okay, so, um, am I worried about it? Yeah, I'm totally worried about it. The, the thing where... The, the narrow component we're talking about now is the existential risk of AI.

Lex Fridman 1:11:00 Ah.

Jeff Hawkins 1:11:01 Right?

Lex Fridman 1:11:01 Yeah.

Jeff Hawkins 1:11:02 So, I wanna make that distinction, 'cause I think AI can be applied poorly, it can be applied in ways that, you know, people are gonna-

Lex Fridman 1:11:09 Sure.

Jeff Hawkins 1:11:09 ... u- u- understand the, the consequences of it. Um, these are all potentially very bad things, but they're not the AI system creating this existential risk on its own. And that's the only place I disagree with other people.

Lex Fridman 1:11:23 Right.

Jeff Hawkins 1:11:23 So.

Lex Fridman 1:11:24 So, I, I think the existential risk thing is, um... Humans are really damn good at surviving. So, to kill off the human race would be very, very difficult.

Jeff Hawkins 1:11:33 Well, you can even... Yes, but you can even... I'll, I'll go further. I don't think AI systems are ever gonna try to. I don't think AI systems are ever gonna like say, "I'm gonna ignore you. I'm gonna do what I think is best." Um, I don't think that's gonna happen, at least not in the way I'm talking about it. So, you... The Twitter, uh, recommendation algorithm is an interesting example. Let's, let's use computers as an analogy again, right? I build a computer. It's a universal computing machine. I can't predict what people are gonna use it for. They can build all kinds of things. They can e- they can even create computer viruses. It's, you know, all kinds of stuff. So, there's some unknown about its utility and about where it's gonna go. But on the other hand, I pointed out that once I build a computer, it's not gonna fundamentally change how it computes. It's like, I used the example of a register, which is a part, internal part of a computer. Um, you know, I say it can't just sit ar- 'cause computers don't evolve. They don't replicate. They don't evolve. They don't... You know, the physical manifestation of the computer itself is not gonna... It... There's certain things it can't do.

Lex Fridman 1:12:35 Yeah.

Jeff Hawkins 1:12:35 Right? So we can... To break into things like, things that are possible to happen we can't predict, and things that are just impossible to happen. They're... U- unless we go out of our way to make 'em happen. They're not gonna happen unless somebody makes 'em happen.

Lex Fridman 1:12:46 Yeah, so there, there's, there's a bunch of things to say. One is the physical aspect, which you're absolutely right. We have to build a thing for it