Lex Fridman PodcastKarl Friston: Neuroscience and the Free Energy Principle | Lex Fridman Podcast #99
EVERY SPOKEN WORD
145 min read · 28,863 words- 0:00 – 1:50
Introduction
- LFLex Fridman
The following is a conversation with Karl Friston, one of the greatest neuroscientists in history, cited over 245,000 times, known for many influential ideas in brain imaging, neuroscience, and theoretical neurobiology, including especially the fascinating idea of the free energy principle for action and perception. Karl's mix of humor, brilliance, and kindness, to me, are inspiring and captivating. This was a huge honor and a pleasure. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcasts, support on Patreon, or simply connect with me on Twitter @Lex Fridman, spelled F-R-I-D-M-A-N. As usual, I'll do a few minutes of ads now, and never any ads in the middle that can break the flow of a conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. When you get it, use code LEXPODCAST. Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. Since Cash App allows you to send and receive money digitally, let me mention a surprising fact related to physical money. Of all the currency in the world, roughly 8% of it is actual physical money. The other 92% of money only exists digitally. So again, if you get Cash App from the App Store or Google Play and use the code LEXPODCAST, you get $10 and Cash App will also donate $10 to FIRST, an organization that is helping to advance robotics and STEM education for young people around the world. And now here's my conversation with Karl Friston.
- 1:50 – 5:53
How much of the human brain do we understand?
- LFLex Fridman
How much of the human brain do we understand from the low level of neuronal communication, to the functional level, to the, uh, to the highest level, maybe the, the psychiatric disorder level?
- KFKarl Friston
Well, we're certainly in a better position than we were last century. (laughs) How far we've got to go, I think is almost an unanswerable question. So you'd have to set the parameters, you know, what constitutes understanding, what level of understanding do you want? I think we've made enormous progress in terms of broad-brush principles. Um, whether that affords a detailed cartography of the functional anatomy of the brain and what it does, and right down to the microcircuitry in the neurons, tha- tha- that's probably, um, out of reach at the present time.
- LFLex Fridman
So the cartography, so mapping the brain, do you think mapping of the brain, the detailed perfect imaging of it, does that get us closer to understanding of the mind, o- of the brain? So how far does it get us if we have that perfect cartography of the brain?
- KFKarl Friston
I think there are lower bounds on that. It's a really interesting question. Um, you, i- and it would determine the sort of scientific career you'd pursue. If you believe that, uh, knowing every dendritic connection, every sort of microscopic synaptic structure right down to the molecular level was gonna give you the right kind of information to understand the computational anatomy, then you'd choose to be a microscopist and you would, um, uh, study little, you know, cubic millimeters of brain for the rest of your life. If, on the other hand, you were interested in holistic functions and, um, a sort of functional anatomy of the sort that a neuropsychologist would understand, you'd study brain lesions and strokes, you know, just looking at the whole person. So again, it comes back to, uh, what level do you want understanding? I think there are principled reasons not to go too far. Um, if you commit to a view of the brain as a machine that's performing a form of inference and representing things, um, there are th- that understanding, that level of understanding is necessarily cast in terms of probability densities and ensemble densities, distributions. And what that tells you is that you don't really want to look at the atoms to understand the thermodynamics of, of probabilistic descriptions for how the brain works. So I personally wouldn't look at the molecules, or indeed the single neurons. In the same way, if I wanted to u- understand the thermodynamics of some non-equilibrium steady state of a gas or an active material, I wouldn't spend my life looking at the, the individual molecules that constitute that ensemble. I'd look at their collective behavior. On the other hand, if you go too coarse-grain, you're gonna miss some basic canonical principles of connectivity and architectures. I'm thinking here, um, it's a bit colloquial, but there's current excitement about high-field magnetic resonance imaging at seven tesla. Why? Well, it gives us, for the first time, the opportunity to look at the brain in action at the level of a few millimeters that distinguish between different layers of the cortex that may be very important in terms of, um, uh, evincing generic principles of co- canonical microcircuitry that are replicated, uh, throughout the brain, that may tell us something fundamental about message passing in the brain and these density dynamics of, or neuronal ensemble population dynamics, uh, that underwrite our, you know, our brain function. So somewhere between a millimeter and a meter.
- 5:53 – 10:43
Most beautiful characteristic of the human brain
- KFKarl Friston
- LFLex Fridman
(laughs) Lingering for a bit on the, on the big questions, if you'll allow me, what to you is the most beautiful or surprising characteristic of the human brain?
- KFKarl Friston
I think its hierarchical and recursive aspect, its recurrent aspect.
- LFLex Fridman
... of the structure or of the actual representational power of the brain?
- KFKarl Friston
Well, I think one speaks to the other. Um, I was actually answering, um, in a dull-minded way from the point of view of purely its anatomy and, uh, and-
- LFLex Fridman
Yeah.
- KFKarl Friston
... its structural aspects. I mean, there are many marvelous organs in the, in the body. Um, let's take your liver, for example.
- LFLex Fridman
(laughs)
- KFKarl Friston
Um, uh, you know, without it, you wouldn't, you wouldn't, um, be around for very long, and it does some beautiful and delicate biochemistry and homeostasis and, you know, evolved with a finesse that would easily parallel the brain. But it doesn't have a beautiful anatomy.
- LFLex Fridman
(laughs)
- KFKarl Friston
It has a simple anatomy, which is attractive in a minimalist sense, but it doesn't have that crafted structure of sparse connectivity and that recurrence and that specialization that the brain has.
- LFLex Fridman
So you s- you said a lot of interesting terms here, so the recurrence, the sparsity, but you also started by saying hierarchical.
- KFKarl Friston
Mm-hmm.
- LFLex Fridman
So I've, uh, I- I've never thought of our brain as hierarchical.
- KFKarl Friston
Uh-huh.
- LFLex Fridman
Uh, w- uh, sort of I always thought it's just like a giant mess-
- KFKarl Friston
(laughs)
- LFLex Fridman
... uh, in- in- interconnected mess where it's very difficult to figure anything out. But, uh, in what sense do you see the brain as hierarchical?
- KFKarl Friston
Well, I see it as it's not a magic soup, um- (laughs)
- LFLex Fridman
Yeah. That's- (laughs)
- KFKarl Friston
... which, of course, is what I used to think when I was, uh, before I, um, uh, studied medicine and the like. Um, so, uh, a lot of those terms imply each other. Um, so hierarchies, uh, if you just think about the nature of a hierarchy, how would you actually build one? And what you would have to do is basically carefully remove the right connections that destroy the, um, completely connected soups that you might have in mind. So a hierarchy is, in and of itself, defined by a sparse and particular connectivity structure. Um, uh, I'm not committing to any particular form of hierarchy, um-
- LFLex Fridman
But your sense is there is some?
- KFKarl Friston
Oh, absolutely, yeah, in virtue of the fact that there is a sparsity of connectivity, not necessarily, um, of a qualitative sort, but certainly of a quantitative sort. So there are, uh, it is demonstrably so and, uh, that the f- far- further apart two parts of the brain are, the less likely tha- they are to be wired, you know, to possess axonal processes, neuronal processes that directly, uh, communicate one message or messages from one part of that brain to the other part of the brain. So we know there's a sparse connectivity, um, and furthermore, on the basis of, um, anatomical connectivity and tracer studies, we know that that, uh, it, that has, has, that sparsity underwrites a hierarch- a hierarchical and very structured sort of, um, um, connectivity that might be best understood like, a little bit like an onion, you know? That there, there is, um, um, a concentric, sometimes referred to as centripetal by people like Marcel Maezlum, hierarchical organization to the brain. So you can think of the brain as, in a rough sense, like an onion, and all the sensory information and all the afferent outgoing messages that supply commands to your muscles or to your secretory organs come from the surface, so there's a massive exchange interface with the world out there on the surface. And then underneath, there's a, a little layer that, um, sits and looks at the exchange on the surface, and then underneath that, there's a layer right the way down to the very center, so the deepest part of the onion. That's what I mean by a, you know, a hierarchical organization. There's a discernible structure defined by the sparsity of c- of connections that lends the architecture, um, a hierarchical structure that tells one a lot about the kinds of representations and messages. So coming back to your earlier question, is this about the representational capacity, uh, or is it about the anatomy? Well, one, um, underwrites the other. Um, you know, if one just simply thinks of the brain as, uh, a message-passing machine, a, a, a, a process that is in the service of doing something, um, then the, the circuitry and the connectivity that shape that message passing also dictate its function.
- LFLex Fridman
So
- 10:43 – 20:38
Brain imaging
- LFLex Fridman
you've done a lot of amazing work in a lot of directions, so le- let's look at one aspect of that, of looking into the brain and trying to study this onion structure. (laughs) Um, what can we learn about the brain by imaging it? Which is one way to sort of look at the anatomy of it.
- KFKarl Friston
Mm-hmm.
- LFLex Fridman
Broadly speaking, what, what are the methods of imaging, but even bigger, what can we learn about it?
- KFKarl Friston
Right, so well, m- most imaging, uh, human neuroimaging that, um, uh, that you might see, um, you know, in science, uh, journals, that speaks to the way the brain works, measures brain activity over time, so, you know, that's the first thing to say, that we're, we're effectively looking at fluctuations in neuronal responses, usually in response to some sensory input or some instruction, um, some task. Uh, not necessarily, and there's a lot of interest in just looking at the brain, um, in terms of resting state endogenous or intrinsic activity. But crucially, at every point, looking at these fluctuations, either induced or intrinsic in neural activity, um, and understanding them at two levels. So, um, normally, people would recourse to two principles of brain oc- organization that are complementary, one functional specialization or segregation. So what does that mean? It simply means that, uh...There are par- certain parts of the brain that may be specialized for certain kinds of processing, you know, for example, visual motion, th- our ability to recognize or to perceive movement in the visual world. And furthermore, that specialized processing may be spatially or anatomically segregated, leading to functional segregation, which means that if I were to compare your brain activity during a period of static, uh, viewing a static image and then compare that to the responses of fluctuations in the brain when you are exposed to a moving image, say, a flying bird, I would, we would expect to see, um, f- restricted, segregated differences in activity. And those are basically the hotspots that you see in the, in such called parametric maps that test for the significance of the responses that are circumscribed. So now, basically, we're talking about what some people have, um, perhaps unkindly called a, a neo-cartography. This is a, a phrenology, um, augmented by modern-day neuroimaging, basically finding, uh, blobs or bumps on the brain that do this or do that and trying to understand the cartography of that functional specialization.
- LFLex Fridman
So how much, how much is there such ... So this is such a beautiful sort of ideal to strive for. Uh, we, uh, we humans, scientists would like, like to s- to hope that there's a beautiful structure to this, whereas, like you said, there's segregated regions that are responsible for the different function. How much hope is there to find such regions in terms of looking at the progress of studying the brain?
- KFKarl Friston
Oh, I think enormous progress has been made, uh, in the past, you know, 20 or 30 years. You know, uh, uh, so this is beyond incremental. You know, at the advent of, uh, brain imaging, the very notion of functional segregation was just a hypothesis based upon a century, if not more, of careful neuropsychology, looking at people who had lost via insult or traumatic brain inmu- um, injury particular parts of the brain and then saying, "Well, they can't do this," or, "They can't do that." For example, losing the visual cortex and not being able to see or using ... losing particular parts of the, uh, visual cortex or, um, uh, regions known as, um, uh, V5 or the middle temporal region, um, MT, and noting that they selectively could not see moving things. And so that created the, um, the, the hypothesis that perhaps movement processing, visual movement processing was located in this functionally segregated area. And you could then put, go and put invasive electrodes in animal models and say, "Yes, indeed, we can excite activity here. We can form, um, receptive fields that are sensitive to or defined in terms of visual motion." But at no point could you exclude the possibility that everywhere else in the brain was also-
- LFLex Fridman
Well-
- KFKarl Friston
... very interested in visual motion.
- LFLex Fridman
By the way, I'm, I apologize to interrupt, but it's a tiny little tangent. You said animal models. Um, just, uh, out of curiosity, from your perspective, how different is the human brain versus the other animals in terms of our ability to study the brain?
- KFKarl Friston
Well, clearly, um, the, the far further away you go from a human brain, the, the, the, the, the greater the differences, but not, not as remarkable as you might think. Uh, so people will choose their level of approximation to the human brain depending upon, you know, the, the kinds of questions that they want to answer. So if you're talking about sort of canonical principles of microcircuitry, it might be f- perfectly okay to look at a mouse. Indeed, you could even look at flies. Uh, worms. Um, if, on the other hand, you wanted to look at the finer details of organization of visual cortex and, um, V1, V2, these are designated sort of patches of cortex that may, uh, may do different things, indeed do, um, you probably want to use, um, a primate that looked a little bit more like a human, um, because there are lots of ethical issues in terms of, you know, the use of, um, non-human primates to, you know, um, to answer questions about the, about, um, human anatomy. But I think most people assume that most of the important principles are conserved in a continuous way, uh, you know, from, um, uh, right from, well, yes, worms right through, (laughs) the whole way through to-
- LFLex Fridman
(laughs)
- KFKarl Friston
... to, to, to, to, to you and me.
- LFLex Fridman
So now returning to this, so that was the early sort of ideas of studying the f- the, the re- functional regions of the brain, whereas if there's some damage to it, you try to infer that there's ... that part of the brain might be somewhat responsible for this type of function. So what ... where does that lead us? What are the next steps beyond that?
- KFKarl Friston
Right. Well, um, well just actually just reverse a bit, come back to your sort of notion that the brain is a magic soup.
- LFLex Fridman
Yeah.
- KFKarl Friston
But that was actually a very prominent idea at, at one point, uh, um, notions such as, uh, Lashley's law of mass action, um, uh, inherited from the observation that, uh, for certain animals, if you just took out spoonfuls of the brain, it didn't matter where you took these spoonfuls out, they always showed the same kinds of deficit. So, you know, it was, it was very difficult to infer functional specialization purely on the base- basis of lesion deficit studies. Um, but once we had the opportunity to look at the brain lighting up and its, its literally, its sort of excitement, neuronal, um, uh, uh, excitement, when looking at this versus that, one was able to say, "Yes, indeed, these functionally specialized responses are very restricted and they, uh, they're here or they're over there. If I do this, then this part of the brain lights up." And that became, um-... doable in the early '90s. Um, in fact, you know, shortly before with the advent of positron emission tomography, and then functional magnetic resonance imaging came along, um, in the early '90s. And since that time there has been an explosion of discovery, refinement, um, confirmation. Um, you know, there are people who believe that it's all in the anatomy. If you understand the anatomy, then you understand the function at some level. And many, many hypotheses were predicated on a, a deep understanding of the anatomy and the connectivity, but they were all confirmed and taken much further, um, with neuroimaging. So that's what I meant by we've made an enormous amount of progress, um, in, in this century indeed, uh, and in relation to the previous century, um, by looking at these functionally selective responses. But that wasn't the whole story. So there was this sort of near phrenology about finding bumps and hotspots in the brain that did this or that. The bigger question was, of course, the functional integration, uh, how all of these, um, regionally specific responses were orchestrated, how they were distributed, how did they relate to distributive processing, and indeed representations in the brain. So then you turn to the, uh, more challenging issue of the integration, the connectivity, and then we come back to this beautiful sparse, recurrent hierarchical connectivity, um, that seems characteristic of the brain and probably not many other, uh, organs.
- LFLex Fridman
And but nevertheless, we come back to this, eh, this challenge of t- trying to figure out how everything is integrated. But what's your feeling? What's the general consensus? Have we moved away from the magic soup view of the brain?
- KFKarl Friston
Yes.
- LFLex Fridman
So there is a deep structure to it.
- KFKarl Friston
Yes.
- LFLex Fridman
That, um... And then maybe d- further question, you said some people believe that the structure is most of it, that you could really get at the core of the function by just deeply understanding the structure.
- KFKarl Friston
Yes.
- LFLex Fridman
Where do you s- sit on that? Do you-
- KFKarl Friston
I think it's got some mileage to it, yes.
- LFLex Fridman
(laughs)
- KFKarl Friston
Yeah. Uh, yeah.
- LFLex Fridman
So it's a worthy pursuit of, of going, um, of, of studying, uh, through imaging and all the different methods to actually study-
- KFKarl Friston
No, absolutely.
- LFLex Fridman
... the structure.
- KFKarl Friston
Yeah, yeah.
- 20:38 – 21:23
Deep structure
- KFKarl Friston
Sorry, I'm just, I'm just noting you, you, you were accusing me of using lots of long words, and then you introduced one there which was deep, which is interesting.
- LFLex Fridman
(laughs)
- KFKarl Friston
Um, 'cause deep is the sort of millennial, um, uh, equivalent of hierarchical. So if you put deep in front of anything (laughs) -
- LFLex Fridman
(laughs)
- KFKarl Friston
... you, you, not only are you very millennial and very trendy, but you-
- LFLex Fridman
That's true.
- KFKarl Friston
... you're also implying a, a hierarchical architecture. So-
- LFLex Fridman
That's true.
- KFKarl Friston
... it is the depth whi- which is, uh, for me the, the beautiful thing.
- LFLex Fridman
That's right. Uh, the word deep kind of, yeah, exactly, it implies hierarchy. I didn't even think about that, that indeed, uh, the implicit meaning of the word deep is, uh, hierarchy.
- KFKarl Friston
Yep.
- LFLex Fridman
Yeah.
- KFKarl Friston
Yep.
- LFLex Fridman
(laughs)
- KFKarl Friston
So deep inside the onion is the center of your soul. That's the-
- LFLex Fridman
(laughs) Beautifully put.
- 21:23 – 32:31
History of brain imaging
- LFLex Fridman
M- maybe briefly, if you could paint a picture of the kind of methods of neuroimaging, maybe the history which you were a part of, you know, from statistical parametric mapping. I mean, just what, what's out there that's interesting for people maybe outside the field that d- to understand o- of what are the actual methodologies of looking inside the human brain?
- KFKarl Friston
Right. Well, there, you can answer that question from two perspectives. Basically, it's the modality, you know, what kind of signal are you measuring, and they can range from, and let's limit ourselves to sort of imaging-based non-invasive, uh, uh, techniques. So you've essentially got brain scanners, and brain scanners can either measure the structural attributes, the amount of water, or the amount of fat, or the amount of iron in different parts of the brain. You can make lots of inferences about, um, the structure of the organ of the sort that you might ab- um, abduce from an X-ray. But a, you know, a very nuanced X-ray that, that is looking at this kinda property or that kinda property. Uh, so looking at the anatomy non-invasively is, would be the first sort of, uh, neuroimaging that people might want to employ. Then you move on to the kinds of measurements that, um, reflect dynamic function. And the most prevalent of those fall into two camps. You've got these, um, metabolic, sometimes hemodynamic blood-related signals.
- LFLex Fridman
Mm-hmm.
- KFKarl Friston
So these metabolic, um, and/or hemodynamic signals are basically proxies for, um, elevated activity, and, message passing, and, and uh, you know, um, neuronal dynamics in particular parts of the brain. Characteristically though, the time constants of these hemodynamic or metabolic responses to neural activity are much longer than the neural activity itself.
- LFLex Fridman
And this is, uh, this is referring... Uh, forgive me for the dumb questions, but this would be referring to blood re- like, the flow of blood?
- KFKarl Friston
Absolutely.
- LFLex Fridman
So-
- KFKarl Friston
Absolutely.
- LFLex Fridman
... there's a ton of, it seems like there's a ton of blood vessels in the brain.
- KFKarl Friston
Yep.
- LFLex Fridman
So but what's the interaction between the flow of blood and the function of the neu- neurons? Is, is there an interplay there, or just-
- KFKarl Friston
Yep, yep. Uh, uh, and that interplay, um, accounts for s- the, uh, several careers of world-renowned scientists. (laughs)
- LFLex Fridman
(laughs)
- KFKarl Friston
So yes, absolutely. Um, so this is known as neurovascular coupling, is exactly what you said. It's how, how does the neural activity, the neuronal infrastructure, the actual message passing that we think, um, um, underlies our capacity to perceive and act, uh, how is that coupled to the vascular responses th- that supply the energy for that neural processing? So there's a delicate web of large vessels, arteries and veins, that gets progressively finer and finer in detail until it perfuses at a microscopic level the machinery where little neurons lie.
- LFLex Fridman
Mm-hmm.
- KFKarl Friston
So coming back to this sort of onion perspective, uh-We were talking before, using the onion as a, as a metaphor for a deep hierarchal structure. But also, I think, it's just an anatomical, anatomically quite a useful metaphor. All the action, all the heavy lifting in terms of neural computation is done on the surface of the brain, and then the interior of the brain is constituted by, um, fatty wires, essentially, axonal processes that are enshrouded by myelin sheaths. And these could be, uh, when you dissect them, they look fatty and white, and so it's called white matter, as opposed to the actual neuropil, which does the computation, constituted largely by neurons, and that's known as gray matter. So the gray matter is a, a, um, a surface or a skin that sits on top of this big ball. Now we are talking magic soup.
- LFLex Fridman
(laughs)
- KFKarl Friston
But a big ball of connections, like spaghetti, very carefully structured with sparse connectivity that preserves this deep hierarchal structure. But all the action takes place on the surface, on the cortex of the onion. Um, and that means that you have to supply the right amount of blood flow, the right amount of nutrient, which is rapidly absorbed and used by neural cells that don't have the same capacity that your leg muscles would have to, um, basically spend their energy budget and then claim it back later. Um, so one peculiar thing about cerebral metabolism, brain metabolism, is it really needs to be driven in the moment, which means you basically have to turn on the taps. So if, um, there's lots of neural activity in one part of the brain, a little patch of a co- few millimeters, even less possibly, you really do have to water that piece of the garden now and quickly. And that, by quickly, I mean within a couple of seconds.
- LFLex Fridman
So that contains a lot of infor- th- that, uh, hence the imaging could tell you a story of what's happening in there.
- KFKarl Friston
Absolutely. But it, it is slightly compromised in terms of the resolution. So the, the deployment of these little micro-vessels that, that water the garden to enable the activity to, to, um, the neural activity to play out, the, the spatial resolution is an order of a few millimeters, um, and crucially, the temporal resolution is the order of a few seconds. So you can't get right down and dirty into the actual spatial and temporal scale of neuronal activity in and of itself. To do that, you'd have to turn to the other big imaging modality, which is the recording of electromagnetic signals as they're generated in real time. So here, the temporal bandwidth, if you like, or the temp, the lower limit on the temporal resolution is, is incredibly small. We're talking about, you know, nanoseconds, milliseconds. And then you can get into, um, the phasic fast responses that is, in and of itself, the neural activity, and, um, start to see the succession or cascade of hierarchal recurrent message passing evoked by a particular stimulus. But the problem is you're looking at electromagnetic signals that have passed through an enormous amount of magic soup or spaghetti of connectivity and through the scalp and the skull. And it's become spatially very diffuse, so it's very difficult to know where you are. So you've got this sort of, um, um, catch-22. You can either use an imaging modality that tells you within millimeters which part of the brain is activated, but you don't know when, or you've got these electromagnetic, uh, EEG, MEG, um, um, setups that tell you to within a few milliseconds when thing, something has responded, but you're not aware. So you've got these two complementary measures, either indirect via the blood flow or direct via the electromagnetic signals caused by neural activity. Uh, these are the two big imaging devices. And then the second level of responding to your question, what, what are the, you know, from the outside, what are the big ways of, of using this, this, this technology? Um, so once you've chosen your, the kind of neuroimaging that you want to use to answer your set questions, and sometimes it would have to be both, um, then you've got a whole raft of analyses, time series analyses usually, that you can, uh, bring to bear in order to answer your questions or address your hypothesis about those data. And, uh, interestingly, they, they both fall into the same two camps we were talking about before, you know, th- this dialectic between specialization and integration, differentiation and integration. So it's the cartography, the blobology analyses.
- LFLex Fridman
I apologize, I probably shouldn't interrupt so much, but just, uh, heard a fun word. Uh, the blah, the-
- KFKarl Friston
Blobology.
- LFLex Fridman
Blobology.
- KFKarl Friston
(laughs) No, it's-
- LFLex Fridman
(laughs)
- KFKarl Friston
... it's a neologism which means the study of blobs.
- LFLex Fridman
(laughs)
- KFKarl Friston
So nothing more. (laughs)
- LFLex Fridman
Uh, are you being, uh, witty and humorous or is there an actual, th- does the word blobology ever appear in a textbook somewhere?
- KFKarl Friston
It would appear in a popular book. Uh, it would not appear in a worthy, um, um, specialist journal.
- 32:31 – 43:05
Neuralink and brain-computer interfaces
- LFLex Fridman
it'd be great if you can sort of comment on... I got a chance recently to spend a day at a company called Neurolink-
- KFKarl Friston
Ah.
- LFLex Fridman
... that, uh, uses brain-computer interfaces, and their dream is to... Well, there's a bunch of sort of dreams, but, uh, one of them is to understand the brain by sort of, um, you know, uh, getting in there past the so-called sort of factory wall, getting in there, be able to listen, communicate both directions. What are your thoughts about this, the future of this kind of technology of brain-computer interfaces to be able to now have a, have a window or direct contact within the brain to be able to measure some of the signals, to be able to send signals, to understand some of the functionality of the brain?
- KFKarl Friston
Ambivalent.
- LFLex Fridman
(laughs)
- KFKarl Friston
My sense is ambivalent. So it's a mixture of good and bad, and I acknowledge that freely. Um, so the good bits, if you just look at the legacy of that kind of, um, reciprocal but invasive, uh, you know, brain stimulation, I didn't paint a complete picture when I was talking about, so the ways we understand the brain prior to neuroimaging. It wasn't just le- lesion deficit studies. Some of the early work, in fact, literally 100 yards from where we're sitting at the Institution of Neurology, was done by, um, stimulating the brain of, say, dogs, and looking at how they responded. Either by- with them, uh, their muscles or with their salivation, and imputing what that part of the brain must be doing, that if I stimulate it, then I, you know, and I evoke this kind of response, then that tells me quite a lot about the functional specialization. So there's a long history of brain stimulation which en- continues to enjoy a lot of attention nowadays, um-
- LFLex Fridman
Positive attention?
- KFKarl Friston
Oh, yes, absolutely. Um, you know, deep brain stimulation for Parkinson's disease is now standard treatment, and also a wonderful vehicle to try and, um, understand the neuronal dynamics that underlie movement disorders like Parkinson's disease. Um, even interest in, um, transme- magnetic, uh, magnetic stimulation, stimulating with magnetic fields, and will it work in people who are depressed, for example. Quite a crude level of understanding what you're doing, but, um, you know, there a- there is historical evidence that these kinds of brute thought interventions do change things, they, you know... It's a little bit like banging the TV when, (laughs) when the valves aren't working properly, but it still, it works. Uh, so, uh, there, you know, there is a long history, uh, uh, brain-computer interfacing or BCI, um, uh, I think is a, is a beautiful example of that. It's sort of carved out its own niche and its own aspirations. Uh, and there have been enormous, uh, advances within limits. Um, um, advances in terms of our ability to understand, um, how the brain, the embodied brain, engages with the world. Um, I'm thinking here of sensory substitution. The augmenting our sensory capacities by giving ourselves extra ways of sensing and sampling the world, um, ranging from sort of trying to replace lost visual signals through to giving people completely new signals. So, uh, um, the, um, one of the m- I think most engaging examples of this is equipping people with a sense of magnetic fields.
- LFLex Fridman
Mm-hmm.
- KFKarl Friston
So you can actually give them magnetic sensors that enable them to feel, should we say, tactile pressure around their tummy-
- LFLex Fridman
Yeah.
- KFKarl Friston
... where they are in relation to the, to, to, to the magnetic field of the Earth.
- LFLex Fridman
That's incredible.
- KFKarl Friston
And a- after a few weeks, they take it for granted. They integrate it, they imbibe, they assimilate-
- LFLex Fridman
That is incredible.
- KFKarl Friston
... this new sensory information into the way that they feel, literally feel their world-
- LFLex Fridman
Yeah.
- KFKarl Friston
... but now equipped with this sense of magnetic di- direction. So that tells you something about the brain's plastic potential to remodel, to, and, uh, uh, um, and its plastic, uh, capacity to suddenly-... try to explain the sensory data at hand by augmentating or augmenting the, uh, the sensory sphere and the kinds of things that, that you can measure. Uh, clearly that's purely for entertainment and, and, and understanding the, you know, the nature and the, and the power of our brains. I would imagine that most BCI is pitched at solving, um, clinical and, um, human problems, such as locked-in syndrome, such as paraplegia, or replacing lost, um, sensory, uh, capacities like blindness and, and deaf- deafness. So, um, then we come to the more um... the negative part of my ambivalence.
- LFLex Fridman
The other side of the-
- KFKarl Friston
The other side of it. (laughs) Um, so I, you know, um, I don't want to be deflationary because much of, uh, my deflationary comments is probably largely out of ignorance than, the- the- the- the- th- than anything else. But, uh, generally speaking, the- the- the- the bandwidth and the bit rates that you get from, um, brain computer interfaces as we currently l- know them, we are talking about bits per second.
- LFLex Fridman
Yeah.
- KFKarl Friston
So that would be like me, um, only being able to communicate with any world or with you using very, very, very slow Morse code. And it is not in the e- in the e- even to within an order of magnitude near what we actually need for an enactive realization of what people aspire to when they think about sort of, um, curing people with paraplegia or replacing sight, um, s- despite heroic efforts. So one has to ask, is there a lower bound on the kinds of, um, recurrent information exchange between a brain and some augmented or artificial, um, um, interface? Um, and then we come back to interestingly what I was talking about before, which is, you know, if you're talking about function in terms of inference, and I presume we'll get to that later on in terms of the free energy principle, but m- at the moment there may be fundamental reasons to assume that is the case. We're talking about ensemble activity. We're talking about basically, um... For example, let's paint the challenge facing, um, brain computer inter- interfacing in terms of controlling another system that is highly and deeply structured, very relevant to our lives, very nonlinear, that rests upon, um, the kind of non-equilibrium steady states and dynamics, uh, that the brain does, the weather. All right? So-
- LFLex Fridman
Good example, yeah.
- KFKarl Friston
... imagine you had some, um, very, um, aggressive satellites that could produce signals that could perturb some little, um, parts of the, um, of the weather system. And then what you're asking now is can I meaningfully get into the weather and change it meaningfully and make the weather respond in a way that I want it to?
- LFLex Fridman
Mm-hmm.
- KFKarl Friston
You're talking about chaos control on a scale which is almost unimaginable. So there may be fundamental reasons why BCI, as you might read about it in a science fiction novel, um, aspirational BCI, may never actually work in the sense that to really be integrated and be part of the system, um, is an im- a requirement that requires you to have evolved with that system, that you, you know, you, um, you have to be part of, um, a very delicately structured, deeply structured dynamic ensemble activity that is not like rewiring a broken computer or plugging in a peripheral interface adapter. It is much more like getting into the weather patterns or e- come back to your magic soup, is g- getting into the active matter, um, and meaningfully relate that to the outside world. So I think there are enormous challenges there.
- LFLex Fridman
So I think, uh, the, the example of the weather is a brilliant one, and I think you paint a really interesting picture. And it wasn't as negative as I thought. It's essentially saying that it's, uh, it might be incredibly challenging, including the low bound of the bandwidth and so on. I kind of... So j- just to full disclosure, I come from the machine learning world. So my, my natural thought is the hardest part is the engineering challenge of controlling the weather, of getting those satellites up and running and, and so on. And once they are, then the rest is a f- fundamentally, uh, the same approaches that allow you to be, to win in the game of Go will allow you to potentially play in this soup, in this chaos.
- KFKarl Friston
Mm-hmm.
- LFLex Fridman
So I have, I have, have a hope that sort of machine learning methods will, will help us play in this soup. Uh, s- but perhaps y- y- you're right that it is a bio- biology and the brain is just an incredible, i- incredible system that, uh, may be almost impossible to get in. But for, for me, what seems impossible is, is the incredible m- mess of blood vessels that you also described without... You know, we also value the brain. (laughs) You can't make any mistakes. You can't damage things. So to me, that engineering challenge seems nearly impossible. One of the things I was really impressed by at Neuralink is, uh, just ha- just, just, just talking to brilliant neurosurgeons and the roboticists.... that, uh, it made me realize that even though it seems impossible, if anyone can do it, it's s- some of these world-class engineers that are trying to take it on. So, um, so I think the conclusion (laughs) of our discussion here is, uh, of- of this part is, uh, is basically (laughs) that the problem is really hard but hopefully not impossible.
- KFKarl Friston
Absolutely.
- 43:05 – 1:24:29
Free energy principle
- LFLex Fridman
let's start with the basics. Uh, so you've also formulated a fascinating principle, the free energy principle. Could we maybe start at the basics, and what is the free energy principle?
- KFKarl Friston
Well, in fact, the free energy principle, um, inherits a lot from, um, the building of these data analytic approaches to these, you know, very high dimensional time series you get, get from the brain. So I think it's interesting to acknowledge that, uh, and in particular, the analysis tools that, um, try to address the other side, which is a functional integration, so the connectivity analyses, um, on the one hand. Uh, but I should also acknowledge it inherits a- an awful lot from machine learning as well. Uh, so, um, uh, the free energy principle, um, is just a formal statement that the, um, the existential imperatives for any system that manages to survive in a changing world is, um, can be cast as, um, a- an inference problem, uh, in the sense that you can interpret the probability of existing as the evidence that you exist. And if you can write down that problem of existence as a statistical problem, then you can use all the maths that has been developed for inference to understand and characterize the ensemble dynamics that must be in play in the service of that inference. So technically, what that means is you can always interpret anything that exists in virtue of being separate from the environment in which it, it exists as trying to, um, minimize variational free energy. And if you're from the machine learning community, you will know that as a negative evidence lower bound or a negative ELBO, um, which is the same as saying you're trying to maximize, or it will look as if all your dynamics are trying to maximize the compliment of that, which is the marginal likelihood or the evidence for your own existence. So that's basically the, you know, the, uh, the free energy principle there.
- LFLex Fridman
But, but to even take a, a, a sort of a small step backwards, you said the existential imperative. (laughs) There's, there's a lot of beautiful poetic words here, but so to put it crudely, it's a f- it's a, it's a fascinating idea of basically des- uh, of trying to describe if you're looking at a blob, how do you know this thing is alive? (laughs)
- KFKarl Friston
Mm-hmm.
- LFLex Fridman
What does it mean to be alive? What does it mean to be, to exist? And so you can look at the brain, you can look at parts of the brain, or you... this is just a general principle that applies to almost any, any, any system, and, uh, uh, that's just a fascinating sort of philosophically, at every level question and a methodology to try to answer that question, what does it mean to be alive?
- KFKarl Friston
Yes.
- LFLex Fridman
(laughs) So, so that, uh, that, that's a huge endeavor, and it's nice that there's at least some, from some perspective, a clean answer. So maybe can you talk about that optimization view of it?
- KFKarl Friston
Yeah.
- LFLex Fridman
So what, what's trying to be minimized and maximized? Wha- a system that's alive, what is it trying to minimize?
- KFKarl Friston
Right. You've- you've made a big move there. Um-
- LFLex Fridman
(laughs)
- KFKarl Friston
... uh, first of all-
- LFLex Fridman
Apologize. (laughs)
- KFKarl Friston
No, no. (laughs) It's- it's good to make big moves. Um, uh, but you- but you've assumed that- that- that- that things, the thing exists bef- the, in a state that could be living or nonliving. Uh, so I may ask you, "Well, what licenses you to say that something exists?" That's why I use the word existential. It- it's beyond living. It's just existence. So if you drill down onto the definition of things that ex- that exist, then they have certain properties if you borrow the maths from non-equilibrium steady state physics that enable you to interpret it, um, their existence, um, in terms of this optimization procedure. So it's good you introduced the word optimization. So, um, what the free energy principle in its sort of, um, most ambitious but also most deflationary and simplest says is that if something exists, then it must, by the mathematics of, um, non-equilibrium steady state, exhibit properties that make it look as if it is optimizing a particular quantity. And it turns out that particular quantity happens to be exactly the same as the evidence lower bound in machine learning, or Bayesian model evidence in Bayesian statistics, or... and then I can list a whole other, you know, list of ways of understanding this, this, this key, um, quantity which, um, is a bound on, on, uh, surprisal, self-information if you're in i- information theory. There are whole, um... there are a number of different perspectives on this quantity. It's just basically the log probability of, uh, being in a particular state. I'm telling this story as an honest, an attempt to answer your question, um, and I'm answering it as if, as if I, uh, uh, um, was pretending to be, uh, a physicist who was-
- LFLex Fridman
(laughs)
- KFKarl Friston
... trying to understand the fundaments of non-equilibrium, um, um, steady state. Um, and I shouldn't really be doing that because the last time I...... uh, was taught physics, I was in my 20s.
- LFLex Fridman
What kind of systems, when you think about the free energy principle, what kind of systems are you imagining? So- so a more specific kind of case study. Uh...
- KFKarl Friston
Yeah. I'm imagining, um, a range of systems, but, uh, y- you know, at its simplest, um, a sim- a single-celled organism, um, that can be identified from its eco-niche or its environment. So, um, so at its simplest, that- that's basically what- what I al- always imagine in my head. And you may ask, "Well, is there any th- how on earth can you even elaborate questions about the existence of a- a sin- a single drop of oil, for example?
- LFLex Fridman
Yes.
- KFKarl Friston
Yeah. What... But there are deep questions there. Why doesn't the oil, why doesn't the thing, the interface between the drop of oil that contains an e- interior and the thing that is not the drop of oil, which is the solvent in which it is immersed, how does that interface persist over time? Why doesn't the oil just dissolve into solvent? Um, so what special properties of the exchange between the surface of the oil drop and the external states in which it's immersed? If you're a physicist, say it would be the heat bath. You know, you've got a, you've got a- a physical system, uh, an ensemble again. We're talking about density dynamics, ensemble dynamics. Um, an ensemble of e- of atoms or molecules immersed in a heat bath. But the question is, how did the heat bath get there and why has it not dissolved? Why-
- LFLex Fridman
How is it maintaining itself?
- KFKarl Friston
Exactly. Yeah.
- LFLex Fridman
What actions... I mean, it's such a fascinating idea of a drop of oil in, I guess it would dissolve in water, it wouldn't dissolve in water. (laughs) So what-
- KFKarl Friston
Precisely. So why not? Uh, so-
- LFLex Fridman
Why not?
- KFKarl Friston
... why not? Yeah.
- LFLex Fridman
And- and how do you mathematically describe... I mean, it's such a beautiful idea. And also the idea of like where does the thing, where does the drop of oil end-
- KFKarl Friston
So yeah. Yep.
- LFLex Fridman
... and where does it begin?
- KFKarl Friston
Right. So I mean, uh, y- you're asking deep questions, deep in- in a non-millennial sense here. (laughs)
- 1:24:29 – 1:28:56
Meaning of life
- LFLex Fridman
We've talked about, uh, living and existence and the objective function under which these objects would operate. What do you think is the objective function of our existence? What- what's the meaning of life?
- KFKarl Friston
(laughs)
- LFLex Fridman
What do you think is the, for you perhaps, the purpose, the source of fulfillment, the source of meaning for your existence as- as one blob-
- KFKarl Friston
As one blob. (laughs)
- LFLex Fridman
... in this soup?
- KFKarl Friston
I'm- I'm- I'm tempted to answer that, uh, again as a physicist in terms-
- LFLex Fridman
(laughs)
- KFKarl Friston
... of the free energy I expect consequent upon my behavior.
- LFLex Fridman
(laughs)
- KFKarl Friston
Um, so technically that, you know, and we could get in a really interesting, uh, conversation about what that comprises in terms of searching for information, resolving uncertainty about the kind of thing that I am.
- LFLex Fridman
Yeah.
- KFKarl Friston
But I th- I suspect that you- you- you- you- you want-
- LFLex Fridman
(laughs)
- KFKarl Friston
... a slightly more personal and fun answer.
- LFLex Fridman
(laughs)
- KFKarl Friston
Um, but which is, can be consistent with that, um, and I think it's, um-... uh, re- reassuringly simple, uh, you, uh, and harps back to the, what you were, um, taught as a child, um, that you have certain beliefs about the kind of creature and the kind of person you are. And all that self-evidencing means, all that minimizing variational free energy in an a- in an inactive and embodied way, means it's fulfilling the beliefs about what kind of thing you are. And of course, we're all given those scripts, those narratives at the very early age, usually in the form of s- bedtime stories or fairy stories, that, um, "I'm a princess and I'm gonna meet a beast who's gonna transform and he's-"
- LFLex Fridman
Yeah.
- KFKarl Friston
"... gonna be a prince." And, um-
- LFLex Fridman
So the narratives are all around you, from your parents, to the, uh, to the friends, to the society, feeds these stories, and then y- then your objective function is to fulfill-
- KFKarl Friston
Exactly.
- LFLex Fridman
... the-
- KFKarl Friston
That narrative that has been encultured by your, your, your immediate family but, you know, as you say, also the sort of the culture in which you, in which you grow up. And you create for yourself. I mean, again, because of this active inference, this inactive aspect of self-evidencing, you know, not only, um, am I modeling my environment, my ekonesh, my, my external states out there, but I'm actively changing them all the time, and external states are doing the same back, we're doing it together. So there's a, a, a synchrony that means that I'm creating my own culture over different te- time scales. So, um, the question now is, for me, being very selfish,
- LFLex Fridman
(laughs)
- KFKarl Friston
... what scripts were I given? It basically was a mixture between Einstein and Sherlock Holmes.
- LFLex Fridman
(laughs)
- KFKarl Friston
So, I'd smoke as heavily as possible-
- LFLex Fridman
(laughs)
- KFKarl Friston
... try to avoid too much interpersonal contact-
- LFLex Fridman
(laughs)
- KFKarl Friston
... um, yet, um, you know, enjoy, um, the, the, uh, the fantasy that, that, th- th- you know, you're, you're a, you're a popular scientist who's gonna make a difference-
Episode duration: 1:29:01
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode NwzuibY5kUs
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome