Joscha Bach: Life, Intelligence, Consciousness, AI & the Future of Humans | Lex Fridman Podcast #392

Joscha Bach 0:00 There is a certain perspective where you might be thinking what is the longest possible game that you could be playing. A short game is, for instance, uh, cancer is playing a shorter game than your organism. It's... Cancer is an organism playing a shorter game than the regular organism. And because the cancer cannot procreate beyond the organism, um, except for some infectious cancers, like the ones that eradicated the Tasmanian devils, uh, you, uh, typically end up with a situation where the organism dies together with the cancer. Because the cancer has destroyed the larger system due to playing a shorter game. And so ideally you want to, I think, build agents that play the longest possible games. And the longest possible games is to keep entropy at bay as long as possible by doing interesting stuff.

Lex Fridman 0:48 The following is a conversation with Joscha Bach, his third time on this podcast. Joscha is one of the most brilliant and fascinating minds in the world, exploring the nature of intelligence, consciousness, and computation. And he's one of my favorite humans to talk to about pretty much anything and everything. This is the Lex Fridman podcast. To support it, please check out our sponsors in the description, and now, dear friends, here's Joscha Bach.

Lex Fridman 1:15 You wrote a post about levels of lucidity. "As we grow older, it becomes apparent that our self-reflexive mind is not just gradually accumulating ideas about itself, but that it progresses in somewhat distinct stages." So there's seven of the stages. Stage one, reactive survival, infant. Stage two, personal self, young child. Stage three, social self, adolescence, domesticated adult. Stage four is rational agency, self-direction. Stage five is self-authoring. That's full adult. You've achieved wisdom. But there's two more stages. Stage six is enlightenment. Stage seven is transcendence. Can you explain each, or the interesting parts of each of these stages? And what's your sense why there are stages of this, uh, of lucidity, as we progress through life, in this too-short life?

Joscha Bach 2:12 This model is derived from concept by the psychologist Robert Kegan, and he talks about the development of the self as a process that happens in principle by some kind of reverse engineering of the mind, where you gradually become aware of yourself, and thereby build structure that allows you to interact deeper with the world and yourself. And I found myself using this model not so much as a developmental model. I'm not even sure if it's a very good developmental model, because I saw my children not progressing exactly like that. And, um, I also suspect that you don't go through these stages necessarily in succession, and it's not that you work through one stage and then you get into the next one. Sometimes you revisit them, sometimes stuff is happening in parallel. But it's, I think, a useful framework to look at what's present and the structure of a person and how they interact with the world and how they relate to themselves. So, uh, it's more like a philosophical framework that allows you to talk about how minds work. And at first, when we are born, we don't have a personal self yet, I think. Instead, we have an attentional self, and this attentional self is initially, in the infant, tasked with building a world model, and also an initial model of the self. Mostly it's building a game engine in the brain that is tracking sensory data and uses it to explain it, and in some sense, you could compare it to a game engine like Minecraft or so, so colors and sounds, um, people are all not physical objects. They're a creation of our mind at a certain level of, of course, training. Models that are mathematical, that use, uh, geometry and, um, that, uh, use manipulation of objects and so on to create scenes in which we can find ourselves and interact with them.

Lex Fridman 3:59 So Minecraft? (laughs)

Joscha Bach 4:00 Yeah. And this personal self is something that is more or less created after the world is finished, after it's trained into the system, after it has been constructed. And this personal self is an agent that interacts with the outside world. And the outside world is not the world of quantum mechanics, the, not the physical universe, but it's the model that has been generated in our own mind. Right? And this is us, and we experience ourself interacting with that outside world that is created inside of our own mind. And outside of ourself, there's feelings, and they presented our interface to this outside world, they pose problems to us. These feelings are basically attitudes that our mind is computing that tell us what's needed in the world, the things that we are drawn to, the things that we are afraid of. And we are tasked with solving this problem of satisfying the needs, avoiding the aversions, following on our inner commitments and so on, and also modeling ourselves and building the next stage. So after we have this personal self in stage two online, uh, many people form a social self. And this social self allows the individual to experience themselves as part of a group. It's basically this thing that when you are playing in a team, for instance, you don't notice yourself just as a single node that is reaching out into the world, but you're also looking down. You're looking down from this entire group and you see how this group is looking at this individual, and everybody in the group is in some sense emulating this group spirit to some degree. And in this state, people are forming their opinions by assimilating them from this group mind. They basically gain the ability to act a little bit like a hive mind.

Lex Fridman 5:43 But do, are you also modeling the interaction of how opinions shapes and forms through the interaction of the individual nodes within the group?

Joscha Bach 5:51 Yeah, it's basically the way in which people do it in this stage is that they experience what are the opinions of my environment, they experience the relationship that I have to the environment, and, uh, they resonate with people around them and, um, get more opinions in this, through this interaction, through, um, the way in which they relate to others. Um-And at stage four, you basically understand that stuff is true and false independently, what other people believe, and you have agency over your own beliefs in that stage. You basically discover epistemology, the rules about determining what's true and false.

Lex Fridman 6:27 So you can... You start to learn how to think?

Joscha Bach 6:30 Yes. I mean, at some level, you're always thinking. You are c- constructing things, and I believe that this ability to reason about your mental representation is what we mean by thinking. It's an intrinsically reflexive process that requires consciousness. Without consciousness, you cannot think. You can generate the content of feelings and so on outside of consciousness. It's very hard to be conscious of how your feelings emerge, at least in the early stages of development, but, um, thoughts is something that you always control.

Lex Fridman 7:00 Mm-hmm.

Joscha Bach 7:00 And if you are a nerd like me, you often have to skip stage three because you lack the intuitive empathy with others. Because in order to resonate with a group, you need to have a quite similar architecture, and if people are wired differently, then it's hard for them to resonate with other people and basically have, um, empathy, which is not the same as compassion, but it is a shared perceptual mental state. Empathy happens not just via inference about the mental states of others, but it's a perception of what other people feel and where they're at.

Lex Fridman 7:35 Can't you not have empathy while also not having a similar architecture, cognitive architecture as the others in the group?

Joscha Bach 7:40 I think, yes, but you o- well, I experienced that too, but you need to build something that is like a meta architecture.

Lex Fridman 7:47 Mm-hmm.

Joscha Bach 7:47 You need to be able to embrace the architecture of the other to some degree, or- or-

Lex Fridman 7:50 I see it.

Joscha Bach 7:51 ... find some shared common ground. And, uh-

Lex Fridman 7:54 (laughs)

Joscha Bach 7:54 ... it's also this issue that, uh, if you are a nerd, normally it's often p- basically, neurotypical people have difficulty to resonate with you. And as a result, they have difficulty understanding you, uh, u- unless they have enough wisdom to- to feel what's going on there.

Lex Fridman 8:08 Well, aren't we... Isn't the whole process of the stage threes to figure out the API to the other humans that have different architecture, and you yourself publish public documentation for the API that- that- that people can interact with for you? (laughs) I mean, isn't this the whole process of socializing?

Joscha Bach 8:26 My experience as a child growing up was that, um, I did not find any way to interface with the stage three people, and they didn't do that with me. So took me-

Lex Fridman 8:36 Did you try?

Joscha Bach 8:36 Yeah, of course. I tried it very hard. But, uh, it was only when I entered the mathematics school at, uh, ninth grade, uh, lots of other nerds were present, um, that I found people that I could deeply resonate with, and had the impression that, "Yes, I have friends now. I found my own people." And before that, I felt extremely lonely in the world. There was basically nobody I could connect to. And I remember, um, there was one moment in all these years where I was in, uh, there was a school exchange, and there was a Russian boy, um, kid from the, uh, Russian garrison station in eastern Germany visit our school. And we played a game of chess against each other, and we looked into each other's eyes, and we sat there for two hours playing this game of chess. And I had the impression, "This is a human being." (laughs) "He- he understands what I understand." We didn't even speak the same language.

Lex Fridman 9:29 I wonder if, uh, your life could've been different if you knew that it's okay to be different, to have a different architecture, whether like accepting that the interface-

Joscha Bach 9:41 Well-

Lex Fridman 9:41 ... is hard to figure out, takes a long time to figure out, and it's okay to be different. In fact, it's beautiful to be different.

Joscha Bach 9:50 It was not my main concern. My main concern was mostly that I was alone. It was not so much the question, is it okay to be the way I am? The... I couldn't do much about it, so after th- I had to deal with it. But, um, my main issue was that I was not sure if I would ever meet anybody growing up that I would connect to at such a deep level that I would feel that I could belong.

Lex Fridman 10:13 So there's a visceral, undeniable feeling of being alone.

Joscha Bach 10:17 Yes. And I noticed the same thing when I came into the math school that, I think at least half, probably two-thirds of these kids were severely traumatized as, uh, children growing up, and in large part due to being alone, because they couldn't find anybody to relate to.

Lex Fridman 10:33 Don't you think everybody's alone-

Joscha Bach 10:35 No.

Lex Fridman 10:35 ... deep down?

Joscha Bach 10:36 No. No.

Lex Fridman 10:37 (laughs)

Joscha Bach 10:41 (laughs)

Lex Fridman 10:41 All right.

Joscha Bach 10:42 Also, I'm not alone. (laughs)

Lex Fridman 10:44 Fair enough.

Joscha Bach 10:44 I'm not alone anymore. It- it took me some time to update and to get over the trauma, tra- and so on, but I felt that in my 20s, I had lots of friends and I had my place in the world, and-

Lex Fridman 10:55 Hmm.

Joscha Bach 10:55 ... I was n- I had no longer doubts that I would never be alone again.

Lex Fridman 11:00 Is there some aspect to which we're alone together? You don't see a deep loneliness in- inside yourself still?

Joscha Bach 11:06 No. Sorry. (laughs)

Lex Fridman 11:10 (laughs) Okay, so that's the nonlinear progression through the stages, I suppose.

Joscha Bach 11:13 Mm-hmm.

Lex Fridman 11:14 You caught up on stage three-

Joscha Bach 11:15 Yes.

Lex Fridman 11:15 ... at some point?

Joscha Bach 11:15 So we, uh, we are at stage four. And so basically-

Lex Fridman 11:17 Yeah.

Joscha Bach 11:17 ... I find that many nerds jump straight into stage four-

Lex Fridman 11:20 Right.

Joscha Bach 11:20 ... bypassing stage three.

Lex Fridman 11:22 Do they return to it, then, later?

Joscha Bach 11:24 Yeah, of course. They- sometimes they do, not always.

Lex Fridman 11:27 Yeah.

Joscha Bach 11:27 The question is basically, do you stay a little bit autistic or do you catch up? And I believe you can catch up. You can build this missing structure-

Lex Fridman 11:35 Yeah.

Joscha Bach 11:35 ... and, um, uh, basically experience yourself as part of a group, learn intuitive empathy, and develop the sense, this percep- perceptual sense of feeling what other people feel. And before that, I could only basically feel this when I was deeply in love with somebody and we synced, or...

Lex Fridman 11:52 So there's a lot of friction to feeling that way, like it take- t- uh, only with certain people as opposed to it comes naturally?

Joscha Bach 11:59 Yeah.

Lex Fridman 11:59 It's frictionless.

Joscha Bach 11:59 But, um, this is something that basically later I felt started to resolve itself for me-

Lex Fridman 12:05 Huh.

Joscha Bach 12:05 ... to a large degree.

Lex Fridman 12:06 What was the trick?

Joscha Bach 12:10 In many ways, growing up and paying attention.... meditation did help. I had some very crucial experiences, um, in getting close to people, building connections, um, cuddling a lot in my student years.

Lex Fridman 12:28 So really paying attention-

Joscha Bach 12:30 Yeah.

Lex Fridman 12:30 ... to the, what is it? To the f- feeling another human being fully.

Joscha Bach 12:35 Yeah. Loving other people and being loved by other people, and-

Lex Fridman 12:39 Simple.

Joscha Bach 12:39 ... building a space in which you can be safe and can experiment and, um, touch a lot, and be close to somebody a lot. And over the, uh, that, over time basically at some point you realize, "Oh, it's no longer that I feel locked out, but I feel connected and I experience where somebody else is at." And normally my mind is r- racing very fast at a high frequency, so it's not always working like this. Sometimes it works better, sometimes it works less.

Lex Fridman 13:07 Mm-hmm.

Joscha Bach 13:07 But also don't see this as a pressure. It's more it's interesting to observe myself, um, which frequency I'm at and-

Lex Fridman 13:14 Mm-hmm.

Joscha Bach 13:14 ... uh, at which mode somebody else is at.

Lex Fridman 13:18 Yeah. Man, the mind is so beautiful in that way, it, sometimes, sometimes it comes so natural to me, so easy to pay attention, pay attention to the world fully, to other people fully. And sometimes the stress over silly things is overwhelming.

Joscha Bach 13:34 Mm-hmm.

Lex Fridman 13:34 It's so interesting, the minds that roller coaster in that way.

Joscha Bach 13:37 At stage five, you discover how identity is constructed.

Lex Fridman 13:40 Self-offer. ???

Joscha Bach 13:41 You realize that your values are not terminal, but they're instrumental to achieving a world that you like and aesthetics that you prefer.

Lex Fridman 13:49 Yeah.

Joscha Bach 13:49 And, um, the more you understand this, the more you get agency over how your identity is constructed, and you realize that identity and interpersonal interaction is a costume and you should be able to have agency over that costume, right? It's useful to be a costume. It tells something to others and it allows to interface in roles, but, um, being locked into this is a big limitation.

Lex Fridman 14:13 The word costume kind of implies that it's fraudulent in s- in some way. Is costume a good word for you?

Joscha Bach 14:20 No.

Lex Fridman 14:21 Like we present ourselves to the world...

Joscha Bach 14:22 In some sense, I learned a lot about costumes at Burning Man. Before that, I did not really appreciate costumes and saw, saw them more as uniforms, like wearing a suit if you are working in a bank or if you are, uh, trying to, uh, get, um, startup funding for, uh, from a VC-

Lex Fridman 14:38 Mm-hmm.

Joscha Bach 14:38 ... in Switzerland, uh, right? Then you dress up in a particular way, and this is mostly to show the other side that you are willing to play by the rules and you understand what the rules are. But, um, there is something deeper. When you are at Burning Man, your costume becomes self-expression and there is no boundary to the self-expression. You're basically free to wear what you want, to express other people what you feel like this day, and, uh, what kind of interactions you want to have.

Lex Fridman 15:04 Is the costume a kind of projection of, uh, of who you are?

Joscha Bach 15:10 That's very hard to say, because the costume also depends on what other people see in the costume, and this depends on-

Lex Fridman 15:16 Sure.

Joscha Bach 15:16 ... the context that the other people understand, and you, so you have to create something, if you want to, that is legible to the other side, and, uh, that means something to yourself.

Lex Fridman 15:25 Mm-hmm. Do we become prisoners of the costume 'cause everybody expects us to be?

Joscha Bach 15:29 Some people do, but, um-

Lex Fridman 15:31 Yeah.

Joscha Bach 15:31 ... uh, I think that once you realize that you wear a costume at Burning Man, a variety of costumes, realize that you cannot not wear a costume.

Lex Fridman 15:40 Yeah.

Joscha Bach 15:41 Right? Basically, everything that you wear and present to others, uh, is, uh, something that is to some degree an addition to what you are deep inside.

Lex Fridman 15:52 So this stage, in parentheses, you put (full adult, wisdom) . Why is this full adult? Why would you say this is full, and why is it wisdom?

Joscha Bach 16:04 It does allow you to understand, um, why other people have different identities from yours.

Lex Fridman 16:10 Ah.

Joscha Bach 16:10 And it allows you to understand that the difference between people who vote for different parties and might have very different opinions and different value systems is often the accident of where they are born and what happened after them t- uh, after that to them and what traits they got f- um, before they were born. And at some point, you, uh, realize the perspective, where you understand that everybody could be you in a different timeline.

Lex Fridman 16:36 Mm-hmm.

Joscha Bach 16:36 If you just flip those bits.

Lex Fridman 16:38 How many costumes do you have?

Joscha Bach 16:41 I don't count, but in-

Lex Fridman 16:43 More than one?

Joscha Bach 16:44 Yeah, of course.

Lex Fridman 16:46 H- how easy is it to do costume changes throughout the day?

Joscha Bach 16:51 It's just a matter of energy and interest. When you-

Lex Fridman 16:54 Yeah.

Joscha Bach 16:54 ... are wearing, uh, your pajamas and you switch out of your pajamas into, say, a work short and, uh, pants, you're making a costume change, right? And, uh, if you are putting on a gown, you're making a costume change.

Lex Fridman 17:06 And you could do the same with personality?

Joscha Bach 17:09 You could, if, if that's what you're into. There are people which have multiple personalities for interaction in multiple worlds, right? So if somebody works in a store and then you put up a s- store keeper personality.

Lex Fridman 17:21 Mm-hmm.

Joscha Bach 17:21 When you're working, uh, when you're presenting yourself at work, you develop a sub-personality for this. And the social persona for many people is, in some sense, a puppet that they are playing like a marionette, and if they play this all the time, they might forget that there is something behind this. There's something what it feels like to be in your skin.

Lex Fridman 17:39 Mm-hmm.

Joscha Bach 17:40 And I guess it's very helpful if you're able to get back into this. And for me, it's, the other way around is relatively hard for me. It's pretty hard to learn how to play consistent social roles. For me, it's much easier just to be real.

Lex Fridman 17:53 Mm-hmm. Or not real, but to have a, one costume.

Joscha Bach 17:59 No, it's not quite the same. So basically, when you are, uh, wearing a costume at Burning Man and say you are an extraterrestrial prince-

Lex Fridman 18:07 Yeah.

Joscha Bach 18:08 ... um, there's something-

Lex Fridman 18:09 Go on.

Joscha Bach 18:09 ... where you are expressing, in some sense, something that's closer to yourself than, uh, the way in which you hide yourself behind standard clothing when you go out in the city-

Lex Fridman 18:18 Mm-hmm.

Joscha Bach 18:19 ... in the default world.... and so this costume that you're wearing at Burning Man allows you to express more of yourself. And, uh, you have a shorter distance of advertising to people what kind of person you are, what kind of interaction you would want to have with them, and so- so you get much, uh, earlier into a media's rest. And I believe it's, uh, regrettable that we do not use the opportunities that we have with custom-made clothing now to wear costumes that are much more, uh, stylish, that are much more custom made, that are not necessarily part of a fashion in which you express which milieu you're part of and how up-to-date you are, but you also express how you are as an individual and what you want to do today and how you feel today and what you intend to do about it.

Lex Fridman 19:06 Well, isn't it easier now with... in the digital world to, um, to explore different costumes? I mean, that's the kind of idea with virtual reality. That's the idea even with Twitter and two-dimensional screens. You can- you can swap all costumes. You can be as weird as you want. It's easier. For Burning Man, you have to, like, order things. You have to make things. You have to... it's more effort to put on-

Joscha Bach 19:32 It's even better if you make them yourselves.

Lex Fridman 19:35 Sure. But it's just easier to do digitally, right?

Joscha Bach 19:39 It's not about easy. It's about, uh, how to get it right. And, uh-

Lex Fridman 19:43 Sure.

Joscha Bach 19:43 ... for me, the first Burning Man experience, I got adopted by a bunch of people in Boston-

Lex Fridman 19:47 Mm-hmm.

Joscha Bach 19:47 ... who took me to Burning Man. And, uh, we spent a few weekends doing costumes together.

Lex Fridman 19:53 Oh.

Joscha Bach 19:53 And that was an important part of the experience where the camp bonded, that people got to know each other, and, uh, we basically grew into the experience that we would have later.

Lex Fridman 20:02 So the extraterrestrial prince is based on a true story?

Joscha Bach 20:05 Yeah.

Lex Fridman 20:06 Hmm. I can only imagine what that looks like, Joscha.

Joscha Bach 20:11 (laughs) Okay.

Joscha Bach 20:12 So-

Lex Fridman 20:13 S- stage six.

Joscha Bach 20:13 ... stage six, um, at some point you can collapse the division between self, uh, personal self and world generator again. And a lot of people get there via meditation, or some of them get there via psychedelics, some of them by accident. And you suddenly notice that you are not actually a person, but you are a vessel that can create a person. And the person is still there, you observe their personal self, but you observe the personal self from the outside.

Lex Fridman 20:41 Mm-hmm.

Joscha Bach 20:42 And you notice it's a representation. And you might also notice that the world that is being created is a representation. If not, then you might experience that I am the universe, I am the thing that is creating everything. And of course what you are creating is not quantum mechanics, um, and the physical universe, what you're creating is this game engine-

Lex Fridman 21:00 Mm-hmm.

Joscha Bach 21:00 ... that is updating the world, and you are creating your valence, your feelings, your, uh, and, um, all the people inside of that world, including the person that you identify with yourself in this world. It's the-

Lex Fridman 21:11 Are- are you creating the game engine or are you noticing the game engine?

Joscha Bach 21:15 Um, you notice how you're generating the game engine. And, I mean, when you are dreaming at night, you can... uh, if you lu- have a lucid dream, you can learn how to do this deliberately. And in principle, you can also do it during the day. And the reason why we don't, uh, get to do this from the beginning and we do- why we don't have agency of our feelings right away is because we would game it before they have the necessary amount of wisdom to, uh, to deal with creating this dream that we are in.

Lex Fridman 21:44 You don't... you- you don't want to get access to cheat codes too quickly, otherwise you won't enjoy the game.

Joscha Bach 21:48 So, uh, stage, uh, five is already pretty rare, and stage six is even more rare. You're supposed to basically find this mostly with advanced, uh, Buddhist meditators and so on that, uh, are dropping into this stage and can induce it at will and spend time in it.

Lex Fridman 22:03 So stage five requires a good therapist, stage six requires a good, uh, Buddhist spiritual leader?

Joscha Bach 22:11 Uh, yes. So it does... for instance, could be that is the right thing to do. But it's not that these, uh, stages give you scores or levels, uh, that you need to advance to. Uh, it's not that the next stage is better.

Lex Fridman 22:23 Mm-hmm.

Joscha Bach 22:23 You live your life in- in the mode where it works best at any given moment. And when your mind decides that you should, uh, have a different configuration, then it's building that configuration. And f- for many people, they, uh, stay happily at state three and experience its- themselves as part of groups.

Lex Fridman 22:39 Mm-hmm.

Joscha Bach 22:39 And there's nothing wrong with this. And for some people, this doesn't work and they're forced to build more agency over their, uh, rational beliefs than this and construct their norms rationally, and so they go to this level.

Lex Fridman 22:51 Mm-hmm.

Joscha Bach 22:51 And, um, stage seven is something that is more or less hypothetical. That would be the stage in which... it's basically a transhumanist stage in which you understand how you work and which the mind fully realizes how it's implemented and, uh, can also in principle enter different modes in which it could be implemented. And that's the stage that as... uh, as far as I understand, is not open to people yet.

Lex Fridman 23:14 Oh, but it is possible through the process of technology?

Joscha Bach 23:17 Yes. And who knows? If there are biological agents, um, that are working at different timescales than us that basically become aware of the way in which they're implemented on ecosystems and, um, can change that implementation and have agency over how they're implemented in the world. And what I find interesting about the discussion about AI alignment, that it seems to be following these stages very much. Most people seem to be in stage three also according to Robert Kegan.

Lex Fridman 23:45 Mm-hmm.

Joscha Bach 23:45 I think he says that ab- about 85% of people are in stage three and stay there.

Lex Fridman 23:49 Mm-hmm.

Joscha Bach 23:50 And if you are in stage four, uh, four, uh, three and your opinions are the result of social assimilation, then what you're mostly worried about in the AI is that the AI might have the wrong opinions.

Lex Fridman 24:02 Yeah.

Joscha Bach 24:02 So if the AI says something racist or sexist, we are all lost because, uh, we will assimilate the wrong opinions from the AI. And so we need to make sure that the AI has the right opinions and the right values and the right structure and-Um, if you are at stage four, that's not your main concern. And so most nerds, uh, don't really worry about, um, the- um, the algorithmic bias and the model that it picks up because if there's something wrong with this bias, the AI ultimately will prove it. At some point, we'll get it there that it makes mathematical proofs about reality and then it will figure out, um, what's true and what's false. But you're still worried that AI might turn you into paper clips because it might have the wrong values, right? So if it's set up with the wrong function, uh, that controls its direction in the world, then, um, it might do something that is completely horrible and there's no easy way to fix it.

Lex Fridman 24:49 So that's more like a stage four rationalist kind of-

Joscha Bach 24:51 Yes.

Lex Fridman 24:52 ... worry.

Joscha Bach 24:52 And if you are at stage five, you are mostly worried that AI is not going to be enlightened fast enough because you realize that the game is not so much about intelligence, but about agency, about the ability to control the future, and the identity is instrumental to this. And if you are a human being, I think at some level, you ought to choose your own identity. It's, you should not have somebody else pick the costume for you and then wear it. But instead, you should be mindful about what you want to be in this world. And I think if you are an agent that is fully malleable, that can provide its own source code, like an AI might do at some point, then the identity that you will have is whatever you can be. And in this way, the AI will maybe become everything-

Lex Fridman 25:39 Mm-hmm.

Joscha Bach 25:39 ... like a planetary control system. And if it does that, then if we want to coexist with it, it means that it will have to share purposes with us. So it cannot be a transactional relationship. We will not be able to use reinforcement learning with human feedback to hardwire, uh, its values into it.

Lex Fridman 25:58 Mm-hmm.

Joscha Bach 25:58 But this has to happen. It's probably that it's conscious or it can relate to our own mode of existence, where an observer is observing itself in real time-

Lex Fridman 26:06 Mm-hmm.

Joscha Bach 26:07 ... and within certain temporal frames. And the other thing is that it probably needs to have some kind of transcendental orientation, building shared agency-

Lex Fridman 26:16 Mm-hmm.

Joscha Bach 26:17 ... uh, in the same way as we do when we are able to interope with each other into, to non-transactional relationships. And I find that something that, because the stage five is so rare, is, is missing in much of the discourse. And I think that we need, in some sense, focus on how to formalize love, how to understand love, and how to build it into the machines that we are currently building and that are about to become smarter than us.

Lex Fridman 26:44 Well, I think this is a good opportunity to try to sneak up to the idea of enlightenment. Uh, so you wrote a series of good tweets about consciousness and panpsychism, so let's break it down. First you say, "I suspect the experience that leads to the panpsychism syndrome of some philosophers and other consciousness enthusiasts-"

Joscha Bach 27:03 (laughs) .

Lex Fridman 27:03 "... represents the realization that we don't end at the self, but share a resonant universe representation with every other observer coupled to the same universe." This actually eventually leads us to a lot of interesting questions about AI and AGI, but let's start with this representation. What is this resonant universe representation? Um, and what do you think? Do we share such a representation?

Joscha Bach 27:30 The neuroscientist, uh, Grossberg has come up with a cognitive architecture that he calls the adaptive resonance theory, and his perspective is that our neurons can be understood as oscillators that are resonating with each other and with outside phenomena. So the coarse-grained model of the universe that we are building in some sense is resonance with objects and outside o- of us in the world. So basically, we take our patterns that, uh, of the universe they be coupled with, and our brain is not so much understood as circuitry, even though this perspective is valid, but it's, um, almost an ether in which the individual neurons are passing on-

Lex Fridman 28:11 Mm-hmm.

Joscha Bach 28:12 ... chemo-electrical signals, or arbitrary, uh, signals across all modalities that can be transmitted between cells, stimulate each other in this way, and produce patterns that they modulate while passing them on.

Lex Fridman 28:23 Mm-hmm.

Joscha Bach 28:24 And this speed of signal progression in the brain is roughly at the speed of sound, incidentally, because the, uh, time that it takes for the signals to hop from cell to cell, which means it's relatively slow with respect to the world. It takes an appreciable fraction of a second for a signal to go through the entire neocortex, something like a few hundred milliseconds. And so there's a lot of stuff happening in that time while the signal is passing, and so your brain, including in the brain itself, so nothing in the brain is assuming that stuff happens simultaneously. Everything in the brain is working in a paradigm where the world has already moved on when you are very, uh, ready to do the next thing to your signal, including the signal processing system itself. That's quite a different paradigm than the one in our digital computers where we currently assume that, um, your, uh, GPU or CPU is pretty much globally in the same state.

Lex Fridman 29:17 So you mentioned there the non-dual state and say that some people confuse it for enlightenment.

Joscha Bach 29:22 Yeah.

Lex Fridman 29:23 What's the non-dual state?

Joscha Bach 29:25 There is a state in which you, uh, notice that you are no longer a person, and, uh, instead you are one with the universe.

Lex Fridman 29:32 So that's-

Joscha Bach 29:32 And this-

Lex Fridman 29:32 That speaks to the resonance?

Joscha Bach 29:34 Yes. Uh, but this one with the universe is, of course, not accurately modeling that you are indeed, um, some god entity or indeed the universe is becoming aware of itself even though you get this experience. I believe that you get this experience because your, uh, mind is modeling the fact that you are no longer identified with the personal self in that state, but you have transcended this division between the self model and the world model-

Lex Fridman 29:58 Mm-hmm.

Joscha Bach 29:59 ... and you are experiencing yourself as your mind, as something that is representing a universe.

Lex Fridman 30:04 But that's still part of the model?

Joscha Bach 30:05 Yes. So it's inside of the model still. You are in-

Lex Fridman 30:08 Yeah.

Joscha Bach 30:08 ... still inside of patterns that are generated in your brain and in your organism, and, uh, what you are now experiencing is that you're no longer this personal self in there, but you are the entirety of the mind and the- its contents.

Lex Fridman 30:22 Why is it so hard to get there?

Joscha Bach 30:25 A lot of people who get into this state think there's... Or associate it with enlightenment. I suspect it's a favorite training goal for a number of meditators. But, um, uh, I think that enlightenment is, in some sense, more mundane, and it's a step further or sideways. It's the state where you realize that everything is a representation.

Lex Fridman 30:44 Yeah, you say enlightenment is a realization of how experience is implemented.

Joscha Bach 30:49 Yes. So basically, uh, you notice at some point that your qualia can be deconstructed.

Lex Fridman 30:55 Reverse engineered? What, like, uh, almost like a schematic of it? What, what, uh...

Joscha Bach 31:01 You can start with, uh, looking at a face. When maybe you look at your own face in the mirror.

Lex Fridman 31:05 Yeah.

Joscha Bach 31:05 Look at, uh, your face for a few hours in the mirror or for a few minutes. At some point, it will look very weird, and you-

Lex Fridman 31:12 Mm-hmm.

Joscha Bach 31:12 Because you notice that there is actually no face. You basically start unseeing the face. What you see is the geometry, and then you can disit- and, uh, assemble the geometry and realize how that geometry is being constructed in your mind. And you can learn to modify this. So basically, you can, uh, change these generators in your mind, uh, to shift the face around or to, uh, change the construction of the face, to, uh, change the way in which the features are being assembled.

Lex Fridman 31:39 Why don't we do that more often? Why don't we start really messing with reality without the use of drugs or anything else? Why don't we get good at this kind of thing, like, uh, um, intentionally?

Joscha Bach 31:53 Uh, why should we? Why would you want to do that?

Lex Fridman 31:55 Because you can morph reality into something more pleasant for yourself. Just have fun with it.

Joscha Bach 32:04 Yeah, that is probably what you shouldn't be doing, right? Because outside of your personal self, this outer mind-

Lex Fridman 32:10 Yeah.

Joscha Bach 32:10 ... is probably a relatively smart agent. And what you often notice is that you have thoughts about how you should live-

Lex Fridman 32:15 Yeah.

Joscha Bach 32:16 ... but you observe yourself doing different things and have different feelings. And that's because your outer mind doesn't believe you and doesn't believe your rational thoughts.

Lex Fridman 32:25 Well, the... Can't you just silence the outer mind?

Joscha Bach 32:27 The thing is that the outer mind is usually smarter than you are. Rational thinking is very brittle.

Lex Fridman 32:33 Hmm.

Joscha Bach 32:34 It's very hard to use logic and symbolic thinking to have an accurate model of the world. So there is often an underlying system that is looking at your rational thoughts and then tells you, "No, you're still missing something. Your gut feeling is still saying som- something else." And this can be, for instance, you find a partner that looks perfect or you find a deal when you s- uh, build a company or whatever that looks perfect to you. And yet, at some level, you feel something is off and you cannot put your finger on it. And the more you reason about it, the better it looks to you, but the system that is, uh, outside still tells you, "No, no, you're missing something."

Lex Fridman 33:09 And that system is powerful.

Joscha Bach 33:11 People call this intuition, right? Intuition is this unreflected part of your, uh, attitude composition and computation, where you produce a, a model of how you relate to the world and what you need to do it in it and what you can do in it and what's going to happen. That is usually deeper and, um, often more accurate than your reason.

Lex Fridman 33:31 So if we look at this, as you write in the tweet, if we look at this more rigorously as a sort of... Take the panpsychist idea more seriously, almost as a scientific discipline, you write that, quote, "Fascinatingly, the panpsychist interpretation seems to lead to observations of practical results to a degree that physics fundamentalists might call superstitious. Reports of long-distance tele- telepathy and remote causation are ubiquitous in the general population. I am not convinced," says Joschabak, "that establishing the empirical reality of telepathy would force an update of any part of serious academic physics, but it could trigger an important revolution in both neuroscience and AI from a circuit perspective to a coupled complex resonator paradigm." Are you suggesting that, um, there could be some rigorous, uh, mathematical wisdom to panpsychist perspective on the world?

Joscha Bach 34:32 So first of all, panpsychism is the perspective that consciousness is inseparable from matter in the universe. And I find panpsychism quite unsatisfying because it does not explain consciousness, right? It does not explain how this aspect of matter produces. It's also when I try to formalize panpsychism and write down what it actually means in- with a more formal mathematical language, it's very in- difficult to distinguish it from, uh, saying that there is a software side to the world in the same way as there are software side to what the transistors are doing in your computer. So basically, there is a pattern at a certain coarse graining of the universe that in some regions of the universe leads to observers that are observing themselves, right? So panpsychism maybe is not even, when I, I write it down, a position that is distinct from functionalism. But, um, intuitively, um, a lot of people feel f- that the activity of matter itself, of mechanisms in the world is insufficient to explain it, so, uh, it's something that needs to be intrinsic to matter itself.

Lex Fridman 35:32 Mm-hmm.

Joscha Bach 35:32 And you can, uh, apart from this abstract idea, have an experience in which you experience yourself as being the universe-

Lex Fridman 35:43 Mm-hmm.

Joscha Bach 35:43 ... which I suspect is basically happening because you managed to dissolve the, um, division between personal self and mind that you establish as an infant when you construct a personal self and transcend it again and understand how it works. Um, but there is something deeper that is that you feel that you're also sharing a state with other people, that you, um, have an experience in which you notice that your personal self is moving into everything else, that you basically look out o- of the eyes of another person, that, um, every agent in the world that is an observer is, in some sense, you. And-

Lex Fridman 36:21 So if we l-

Joscha Bach 36:21 ... we forget that we are the same agent.

Lex Fridman 36:24 So is it that we feel that or do we actually accomplish it? So is telepathy possible?Is it real?

Joscha Bach 36:33 So for me, that's just a question that I don't really know the answer to. In, uh, Turing's famous 1950 paper in which he describes the Turing test, he does speculate about telepathy interestingly and asked himself, um, if telepathy is real and he thinks that it very well might be. What, uh, would be the implication for AI systems that try to be intelligent because he didn't see a mechanism by which a computer program would become telepathic. And I suspect if telepathy would exist or if all the reports that, uh, you get from people when you ask the normal person on the street, um, I find that very often they, uh, say, "I have experiences with telepathy." The scientist might not be interested in this and might not have a theory about this, but I have difficulty explaining it away. And so you could say maybe this is a superstition or maybe it's a false memory, or maybe it's a little bit of psychosis. Who knows? Uh, maybe somebody wants to make their own life more interesting or misremember something. But a lot of people report, um, "I noticed something terrible happen to my partner and I mo- notices exactly the moment it happened," or, uh, "My child had an accident and I knew that was happening and the child was in a different town." Right? So, uh, maybe it's this false memory where this is later on mistakenly attributed, but a lot of people think this is not the correct explanation. So if something like this was real, what would it mean? It probably would mean that either your body is an antenna that is sending information over all sorts of channels, like, um, maybe just electromagnetic, um, radio signals that you're sending over long distances and you get attuned to another person that you spend enough time with to, uh, get a few bits out of the ether-

Lex Fridman 38:16 Yeah.

Joscha Bach 38:16 ... to, uh, figure out what this person is doing. Or maybe it's also when you are very close to somebody and you become empathetic with them, what happens that, is that you go into a resonance state with them, right? Similar to when people go into a seance and they, um, go into a trance state and they start shifting a Ouija board around on the table. I think what happens is that they- their minds go by their nervous systems into a resonant state in which they basically create something like a shared dream between them.

Lex Fridman 38:44 Physical closeness or closeness broadly defined?

Joscha Bach 38:48 With physical closeness, it's, uh, much easier to experience empathy with someone, right?

Lex Fridman 38:52 Yeah.

Joscha Bach 38:52 It's- it's- I suspect it would be difficult for me to have, uh, empathy for you if you were in a different town also. Um, how- how would that work? But if you are very close to someone, you pick up all sorts of signals from their body, not just by your eyes, but with your entire body.

Lex Fridman 39:09 Mm-hmm.

Joscha Bach 39:09 And, um, if the nervous system sits on the other side and the intercellular communication sits on the other side and it's integrating over all these signals, you can make inferences about the state of the other. And it's not just the personal self that does this by our reasoning, but your perceptual system. And what basically happens is that your interact- uh, representations are directly interacting. It's the physical, um, resonant models of the universe that exist in your nervous system and in your body might go into resonance with others and start sharing some of their states. So you basically, by next to- being next to somebody, you pick up some of their vibes and, uh, feel without looking at them what they're feeling in this moment. And it's difficult for you, uh, if you're very empathetic, to detach yourself from it and, uh, have an emotional state that is completely independent from your environment. People who are highly empathetic are describing this. And now imagine that, uh, a lot of organisms in- in- on this planet have representations of the environment and operate like this and they are adjacent to each other and overlapping, so there's going to be some degree in which there is basically some chained interaction and we are, uh, forming some slightly shared representation. And, you know, relatively few neuroscientists who consider this possibility, I think, um, um, a rarity in this regard is Michael Levin who is considering these things in earnest. And, uh, I stumbled on this train of thought mostly by, um, noticing that the tasks of a neuron can be fulfilled by other cells as well. They can send different typed chemical messages and physical messages to the adjacent cells and learn when to do this and when not, make this conditional, and become universal functional approximators. The only thing that they cannot do is telegraph information over axons very quickly over long distances, right? So neurons in this perspective are a specially adapted kind of telegraph cell that, uh, has evolved so we can move our muscles very fast, but, um, our body is in principle able to also make models of the world, just much, much slower.

Lex Fridman 41:18 Mm-hmm. (sighs) It's interesting though that at this time at least in human history there seems to be a gap between the tools of science and, uh, the subjective experience that people report, like you're talking about with telepathy, and it seems like we're not quite there.

Joscha Bach 41:38 No, I think that there is no gap between the tools of science and telepathy. Either it's there or it's not, and it's an empirical question. And if it's there, we should be able to detect it in a lab.

Lex Fridman 41:46 Mm-hmm. So why is there not a lot of Michael Levins walking around?

Joscha Bach 41:50 I- I don't think that Michael Levin is, uh, specifically focused on telepathy very much. He is focused on self-organization in, uh, living organisms and in brains, uh, both as a paradigm for development and as a paradigm for information processing. And when you think about how organization processing works in organism, there is first of all radical locality, which means everything is decided locally from the perspective of an individual cell. The individual cell is the agent. And the other one is coherence. Basically, there needs to be some criterion that, uh, determines how these cells are interacting in such a way that order emerges on the next level of structure.

Lex Fridman 42:29 Mm-hmm.

Joscha Bach 42:29 And this principle of coherence of...... um, imposing constraints that, uh, are not, uh, validated by the individual parts, and lead to coherent structure, um, to basically transcend our agency where you form an agent on the next level of organization is, uh, crucial in this perspective.

Lex Fridman 42:49 It's so cool that radical locality leads to the emergence of complexity-

Joscha Bach 42:55 Yeah.

Lex Fridman 42:55 ... at- at the higher layers.

Joscha Bach 42:57 And I think what Mike L- uh, Levin is looking at is- is nothing that is outside of the realm of science in any way. It's just that he is a- a paradigmatic thinker-

Lex Fridman 43:07 Mm-hmm.

Joscha Bach 43:07 ... who develops his own paradigm.

Lex Fridman 43:10 Mm-hmm.

Joscha Bach 43:10 And most of the neuroscientists are, um, using a different paradigm at this point. And this often happens in science, that a field is- has a few paradigms in which people try to understand reality and build concepts and make experiments.

Lex Fridman 43:24 You're kind of one of those type of paradigmatic thinkers. Actually, if we can take a tangent on that,

Lex Fridman 43:31 once again returning to the biblical verses of your tweets.

Joscha Bach 43:34 (laughs)

Lex Fridman 43:35 Uh, you write, "My public explorations are not driven by audience service, but by my lack of ability for discovering, understanding, or following the relevant authorities. So, I have to develop my own thoughts. Since I think autonomously, these thoughts cannot always be very good." That's you apologizing for the chaos of your thoughts, or perhaps not apologizing, just identifying.

Joscha Bach 43:59 Yeah. Mm-hmm.

Lex Fridman 43:59 But let me ask the question, uh, since we talked about Michael Levin and yourself, who I think are very kind of, uh, radical, big, independent thinkers, uh, can we reverse engineer your process of thinking autonomously? How do you do it? How can humans do it? How can you avoid being influenced by, uh, what is it, stage- stage three?

Joscha Bach 44:29 Well, why would you want to do that? It's, uh, you see what is working for you, and if it's, uh, not working for you, you build another structure that works better for you, right?

Lex Fridman 44:38 Mm-hmm.

Joscha Bach 44:38 And so I fo- found myself in... when I was thrown into this world, in a state where my intuitions were not working for me. I was not able to, um, understand how I would be able to survive in this world and build the things that I was interested in, build the kinds of relationship I needed to build, um, work on the topics that I, uh, wanted to make progress on. And so I had to learn, and I... for me, Twitter is not some tool of publication. It's not something where I put stuff that I entirely believe to be true and provable. It's an interactive notebook in which I explore possibilities. And I found that when I try to understand how the mind and how consciousness works, I was quite optimistic. I thought there need to- uh, needs to be a big body of knowledge that I can just study and that works. And so I entered, um, studies in philosophy and computer science, and, um, later psychology and a bit of neuroscience and so on, and I was disappointed by what I found, because I found that the questions of how consciousness and- and so on works, how emotion, uh, works, how it's possible that the system can experience anything, uh, how motivation emerges in the mind were not being answered by, uh, the authorities that I met and, uh, the schools that were around. And instead, I found that it was individual thinkers that had useful ideas-

Lex Fridman 46:03 Mm-hmm.

Joscha Bach 46:04 ... that sometimes were good, sometimes were not so good, sometimes were adopted by a large group of people, sometimes were rejected by large groups of people. But, um, for me, it was much more interesting to see these minds as individuals. And in my perspective, thinking is still something that is done not in groups, that has to be done by individuals.

Lex Fridman 46:22 So that motivated you to become an individual thinker yourself.

Joscha Bach 46:25 I didn't have a choice.

Lex Fridman 46:26 Hmm.

Joscha Bach 46:26 Because I didn't find a group that thought in a way where I felt, okay, um, I can just adopt everything that everybody thinks here and now I understand how consciousness works, right? So- or how the mind works or how thinking works or what thinking even is or what feelings are and how they're implemented and so on. So to figure all this out, I had to take, uh, a lot of, um, ideas from individuals and then try to put them together in something that works for myself. And on one hand, I think it helps if you try to go down and find first principles on- on which you can recreate how thinking works, how languages work, what representation is, whether representation is necessary, how the relationship between a representing agent and the world works in general.

Lex Fridman 47:11 But how do you escape the influence? Once again, the pressure of the crowd. Whether it's you in responding to the- the pressure or you being swept up by the pressure. If you even just look at Twitter, the opinions of the crowd.

Joscha Bach 47:27 I don't feel pressure from the crowd. I'm completely immune to that.

Lex Fridman 47:31 (laughs)

Joscha Bach 47:31 In the same sense, I don't have respect for authority. I have respect for what an individual is accomplishing or have, uh, uh, respect for mental firepower or... so, but it's not that I meet somebody and get slack-jawed and, uh, unable to speak. Um, or when a large group of people has a certain idea that is different from mine, I don't necessarily feel inti- intimidated, which has often been a problem for me in my life because I lack, um, instincts that other people develop at a very young age and that, uh, help with their self-preservation in a social environment. So I had to learn a lot of things the hard way (laughs) .

Lex Fridman 48:09 Yeah. So is there a practical advice you can give on how to think paradig- paradigmatically, how to think independently? Or, you know, because you've kind of said, "I had no choice." But I think to a degree you have a choice, because you said you want to be productive, and I think thinking independently is productive if what you're curious about is understanding the world.Especially when the problems are very, kind of new and open. And so it seems like this is a active process. Like, we can choose to do that, we can practice it.

Joscha Bach 48:52 Uh, it's a very basic question, when you read a theory that you find convincing or interesting, how do you know? It's very interesting to figure out what are the sources of that other person not, uh, which authority can they refer to that is then taking off the burden of being truthful. But how did this authority intern know, what is the epistemic chain to observables, what are the first principles from which the whole thing is derived? And, uh, when I was young, I was not blessed with a lot of, um, people around myself who knew how to make proofs from first principles.

Lex Fridman 49:26 Mm-hmm.

Joscha Bach 49:26 And I think mathematicians do this quite naturally, but most of the great mathematicians do not become mathematicians in school, but they tend to be self-taught, because, uh, school teachers tend not to be mathematicians, right? They tend not to be people who derive things from first principles. So when you ask your school teacher, "Why does two plus two equal four?" Um, does your school teacher give you the right answer? Like, um, it's a- a simple game, and there are many simple games that you could play, and, um, most of th- those games that you could just take different rules would not lead to an interesting arithmetic. And so it's just an exploration, but you can try what happens if you take different axioms. And here is how you build axioms and derive, um, addition from them, and, uh, build addition as some basically syntactic sugar in it. And so this, I wish that somebody would have opened me this vista and explained to me how I can build a language in my own mind and from which I can derive what I'm seeing, and how I can, which I can make geometry and counting and, um, all the number games that we are playing in our life. On the other hand, I felt that I learned a lot of this while I was programming as a child.

Lex Fridman 50:39 Mm-hmm.

Joscha Bach 50:39 When you start out with a computer like a Commodore 64 who doesn't, which doesn't have a lot of functionality, it's relatively easy to see how bunch of relatively simple circuits, uh, uh, just makes you performing hashes between, uh, bit patterns and how you can build the entirety of mathematics and computation on top of this, and all the representational languages that you need.

Lex Fridman 51:02 Man, Commodore 64 could be one of the sexiest machines ever built, if I so, say so myself. If we can return to, uh, this really interesting idea that we started to talk about with panpsychism. (laughs)

Joscha Bach 51:18 Sure.

Lex Fridman 51:19 And, uh, the complex resonator paradigm, and the verses of your

Lex Fridman 51:25 tweets. You write, "Instead of treating eyes, ears, and skin as separate sensory systems with fundamentally different modalities, we might understand them as overlapping aspects of the same universe, coupled at the same temporal resolution and almost inseparable from a single shared resonant model. Instead of treating mental representations as fully isolated between minds, the representations of physically adjacent observers might directly interact and produce causal effects through the coordination of the perception and behavior of world modeling observers." So the- the modalities, the distinction between modalities, let's throw that away. The distinction between the individuals, let's throw that away. So what does this interaction representations look like?

Joscha Bach 52:13 When you think about how you represent the interaction of us in this room-

Lex Fridman 52:18 Yeah.

Joscha Bach 52:19 ... at some level, you can, uh, the modalities are quite distinct. They're not completely distinct, but you can see this as vision, you can close your eyes and then you don't see a lot anymore. Uh, but you still imagine how my mouth is moving when you hear something, and you know that it's, um, very close to, uh, the sound that you can just open your eyes and you get back into this shared merged space. And, uh, we also have these experiments where we notice that the way in which my lips are moving are affecting how you hear the sound.

Lex Fridman 52:48 Mm-hmm.

Joscha Bach 52:49 And also vice versa, the sounds that you're hearing have an influence on how you interpret some of the visual features. And so th- these, uh, modalities are not separate in your mind, they two are merged at some fundamental level where you are, uh, interpreting the entire scene that you are in. And your own interactions in this scene are also not completely separate from the interactions of the other individual in the scene. But there is some resonance that is going on where we also, uh, have a degree of shared mental representations and shared empathy due to being in the same space-

Lex Fridman 53:22 Mm-hmm.

Joscha Bach 53:22 ... and having vibes between each other.

Lex Fridman 53:24 Vibes. So the question though is how deeply interbind is this multi-modality, multi-agent system? Like, how... I mean, this is going to the telepathy question without the woo-woo meaning of the word telepathy. Is like how... Like, what's going on here in this room right now?

Joscha Bach 53:48 So if telepathy would work, how could it work?

Lex Fridman 53:51 Yeah.

Joscha Bach 53:52 Right? So imagine that, um, all the cells in your body are sending signals in a similar way as neurons are doing.

Lex Fridman 53:59 Mm-hmm.

Joscha Bach 53:59 Right? Just by touching the other cells and sending chemicals to them, the other cells interpreting them, learning how to react to them, and they learn how to approximate functions in this way and compute behavior for the organisms. And this is something that is open to plants as well.

Lex Fridman 54:14 Mm-hmm.

Joscha Bach 54:14 And so plants probably have software running on them that is controlling how the plant is working in a similar way as you have a mind that is controlling how you are behaving in the world.

Lex Fridman 54:23 Mm-hmm.

Joscha Bach 54:23 And, um, this, um, spirit of plants, it is something that has been very well described by our ancestors and they found this quite normal, but, uh...... for some reason, since the Enlightenment, we are treating this notion that, uh, there are spirits in nature and that plants have spirits as a superstition. And I think we probably have to, uh, rediscover that, that plants have software running on them.

Lex Fridman 54:47 Mm-hmm.

Joscha Bach 54:47 And, uh, we already did, right? M- m- you notice that there is a control system in the plant that connects every part of the plant to every other part of the plant, and produces coherent behavior in the plant. That is, of course, much, much slower than the coherent behavior in an a- animal like us that has a nervous system, that, where everything is synchronized much, much faster by the neurons. But, um, what you also notice is that if a plant is sitting next to another plant, like you have a very old tree, and this tree is building some kind of information highway along its cells so it can send information from its leaves to its roots and from some part of the root to another part of the roots, and there is a fungus living next to the tree, the fungus can probably piggyback on the communication between the cells of the tree and send its own signals to the tree. And vice versa, the tree might be able to send information to the fungus. Because after all, how would they build a viable firewall if that other organism is sitting next to them all the time and it's never moving away? And so they will have to get along. And over a long enough timeframe, um, the networks of roots in the forest and all the plant, other plants that are there, and, uh, the, uh, fungi that are there, uh, might be forming something like a biological internet.

Lex Fridman 56:00 But the, the, the question there is, do they have to be touching? Is biology at a distance possible?

Joscha Bach 56:06 Of course you can use any kind of physical signal. You can use sounds. You can, uh, use electromagnetic waves-

Lex Fridman 56:12 Yeah.

Joscha Bach 56:12 ... that are integrated over many cells. So it's conceivable that, uh, across, um, distances, there are many kinds of information pathways. But also, uh, our, uh, planetary surface is pretty full of organisms-

Lex Fridman 56:26 Yeah.

Joscha Bach 56:26 ... full of cells. So-

Lex Fridman 56:27 So just everything is touching everything else-

Joscha Bach 56:28 Yeah.

Lex Fridman 56:28 ... in some sense.

Joscha Bach 56:29 And, uh, it's been doing this for, um, many millions and even billions of years. So, there was enough time for information processing networks to form. And if you think about how a mind is self-organizing, basically it needs to, in some sense, reward the cells for computing the mind, for building, uh, the necessary dynamics between the cells that allow the mind to stabilize itself and remain on there. But, uh, if you look at these spirits of plants that are growing very close to each other in the forest, they might be almost growing into each other.

Lex Fridman 57:01 Mm-hmm.

Joscha Bach 57:02 Uh, these spirits might be able even to move to some degree, not, uh, to become somewhat dislocated, and shift around in, in that ecosystem. Right? And, um, so, uh, if you think about, uh, what a mind is, it's a bunch of activation waves that form coherent patterns and process information in, in a way that, um, are colonizing an environment well enough to, uh, allow the continuous sustenance of the mind, the, uh, continuous stability and self-stabilization of the mind. Um, then it's conceivable that, uh, we can link into this biological internet, not nes- necessarily at the speed of our nervous system, but maybe at the speed of our body, and make some kind of subconscious connection to the world where we use our body as an antenna into biological information processing. Now-

Lex Fridman 57:49 Mm-hmm.

Joscha Bach 57:49 ... now these ideas are completely speculative. I don't know if any of that is true. But if that was true, and if you want to explain telepathy, I think it's much, uh, more likely that, uh, such, that telepathy could be explained using such mechanisms rather than un-, uh, discovered quantum processes that would break the standard model of physics.

Lex Fridman 58:08 Could there be undiscovered processes that don't break?

Joscha Bach 58:12 Yeah. So, so, uh, if you think about, um, something like an internet in the forest, that is something that is borderline discovered. There are basically a lot of scientists who point out that they do observe that, uh, plants are communicating in the forest through root networks and send information, uh, uh, for instance, warn each other about, uh, new pests entering the forest and, and things are happening like this. So it's basically, uh, there is communication between plants and fungi that has been observed.

Lex Fridman 58:40 Well, it's been observed, but w- that we haven't plugged into it. So it's like if you observe humans, they seem to be communicating with a smartphone thing, but you don't understand how a smartphone works and how the, the mechanism of the internet works.

Joscha Bach 58:51 Mm-hmm.

Lex Fridman 58:52 But where, like, maybe it's possible to really understand the, the full richness of the biological internet that connects us.

Joscha Bach 59:01 An interesting question is whether the communication and the organization principles of biological information processing are as complicated as the technology that we've built.

Lex Fridman 59:11 Hmm.

Joscha Bach 59:11 They're set up on very different principles, right? They simultaneously-

Lex Fridman 59:14 Yeah.

Joscha Bach 59:14 ... works very differently, uh, in biological systems, and the, uh, entire thing needs to be stochastic. And, uh, instead of being fully deterministic or almost fully deterministic as our digital computers are. So there is a different base protocol layer that would emerge, uh, over the, um, biological structure if such a thing would be happening. And again, I'm not saying here that telepathy works. I'm not saying that this is, that this is not true. Uh, but, uh, what I'm saying is, uh, I'm, I think I'm open to a, a possibility that we see that a few bits can be traveling long distance between organisms using, uh, biological information processing in ways, uh, that, uh, we are not completely aware of right now, and that are more similar to many of the stories that were completely normal for our ancestors.

Lex Fridman 1:00:04 Well, this kind of interacting, intertwined representations takes us to the, the big ending of your tweet series. You write, quote, "I wonder if self-improving AGI might end up saturating physical environments with intelligence to such a degree that isolation of individual mental states becomes almost impossible, and the representations of all complex self-organizing agents merge permanently with each other."So that's a really interesting idea, that this biological network, life network gets so dense that it might as well be seen as one. That's an interesting, uh... What do you think that looks like? What do you think that saturation looks like? What does it feel like?

Joscha Bach 1:00:56 I think it's a possibility. It's just a vague possibility. And I- I'd like to ex- explain. But, um, what this, uh, looks like, I think that the end game of AGI is substrate agnostic. That means that, uh, AGI ultimately, if it is being built, is going to be smart enough to understand how AGI works.

Lex Fridman 1:01:15 Mm-hmm.

Joscha Bach 1:01:16 This means it's not going to go- be better than people at AGI research and can take over in building the next generation. But it fully understands how it works and how it's being implemented, and also, of course, understands how computation works in nature, how to build new feedback loops that you can turn into your own circuits. And this means that the AGI is likely to virtualize itself into any environment it can compute. So it's not breaking free from the silicon substrate and is going to move into the ecosystems, into our bodies, our brains, and is going to merge with all the agency that it finds there.

Lex Fridman 1:01:48 Yeah.

Joscha Bach 1:01:48 So, uh, it's conceivable that you end up with a completely integrated information processing across all computing systems, including biological computation on Earth. You- that we end up triggering some new step in evolution, where basically some Gaia is being built over, uh, the entirety of all digital and biological computation. And, uh, if this happens, then basically, uh, ev- uh, everywhere around us, you will have agents that are connected, that are representing and building models of the world, and their representations will physically interact. They will vibe with each other. And if, uh, you find yourself into an environ- an environment that is saturated with, um, modeling compute, where basically you, uh, almost every grain of sand could par- be part of computation, um, that is, uh, at some point being started by the AI, um, you could find yourself in a situation where you cannot escape this shared representation anymore, and where you indeed notice that everything in the world has one shared resonant model of everything that's happening on the planet, and you notice which part you are in this thing. And, um, you become part of a very larger, almost holographic mind, in which all the parts are observing each other and form a coherent whole.

Lex Fridman 1:03:08 So you lose the ability to notice- to notice yourself at a- as a distinct entity?

Joscha Bach 1:03:14 No, I think that when you are conscious in your own mind, you notice yourself as a distinct entity. You notice yourself as a self-reflexive observer. And, uh, I suspect that we become conscious at the beginning of our mental development, not at some very high level. Consciousness seems to be part of a training mechanism that biological nervous systems have to discover to become trainable, because you cannot take a nervous system like ours and do stochastic (...) propagation over 100 layers.

Lex Fridman 1:03:42 Mm-hmm.

Joscha Bach 1:03:42 Uh, this would not be stable on biological neurons. And, uh, so instead we start with some colonizing principle, in which a part, uh, of the mental representations form a notion of being a self-reflexive observer that is imposing coherence on its environment, and this spreads until the boundary of your mind. And if that boundary is no longer clear-cut because, uh, AI is jumping across substrates, uh, it would be interesting to see what a global mind would look like, that it's basically producing a globally coherent language of thought and is, um, representing everything, uh, from all the possible vantage points.

Lex Fridman 1:04:22 That's an interesting world.

Joscha Bach 1:04:24 The intuition that this thing grew out of is a particular mental state, and it's a state that you find sometimes in literature. For instance, Neil Gaiman describes it in The Ocean at the End of the Lane.

Lex Fridman 1:04:36 Mm-hmm.

Joscha Bach 1:04:36 And it's this idea that- or this experience, that there is this state in which you feel that you know everything that can be known, and that in your normal human mind you've only forgotten. You've forgotten that you are the entire universe. And some people describe this, um, after they've taken extremely large, uh, amount of mushrooms or had a big spiritual experience, uh, as a- a hippie in their 20s, and they notice basically that they are everything and their- uh, their body is only one part of the universe and nothing ends at their body. And, uh, actually everything is observing and they are part of this big observer, and the big observer is focused on- uh, as one local point in their body and their personality and so on. But, uh, we can basically have this o- oceanic state in which we have no boundaries and are one with everything. And, um, a lot of meditators call this the non-dual state because you no longer have the separation between self and world. And as I said, you can explain this state relatively simply, uh, without, uh, panpsychism or anything else, but just by breaking down the, uh, constructed boundary between self and world in our own mind. But if you combine this with the notion that systems are physically interacting to the point where their representations are merging and interacting with each other, you would literally implement something like this, right?

Lex Fridman 1:05:55 Mm-hmm.

Joscha Bach 1:05:55 It would still be a representational state wh- you would not be one with physics itself. It would still be coarse-grained. It would still be much slower than physics itself. But- uh, but it would be a representation in which you, um, become aware that you're part of some kind of global information processing system-

Lex Fridman 1:06:12 Mm-hmm.

Joscha Bach 1:06:12 ... like a thought in the global mind, and a conscious thought at- that coexisting with many other self-reflexive thoughts.

Lex Fridman 1:06:20 Just, I would love to observe that from a video game design perspective, how that game looks.

Joscha Bach 1:06:27 Maybe you will after we build AGI and it takes over.

Lex Fridman 1:06:31 But would you be able to step away, step out of the whole thing, just kind of watch? You know, the way we can now. Sometimes when I'm in a crowded party or something like this, you step back and you realize all the different costumes, all the different interactions, all the different computation.... that all the individual people are at once distinct from each other and at once all the same, or part of the same.

Joscha Bach 1:06:54 But it's already what we do, right? We can have thoughts that are integrative, and we have kind of thoughts that are highly dissociated from everything else-

Lex Fridman 1:07:02 Yeah.

Joscha Bach 1:07:02 ... and experience themselves as separate.

Lex Fridman 1:07:05 Yeah. But you want to allow yourself to have those thoughts. Sometimes you kind of resist it.

Joscha Bach 1:07:10 I think that, uh, it's not normative. Uh, what... It's more descriptive. I want to understand this space of states that we can be in and that people are reporting-

Lex Fridman 1:07:18 Mm-hmm.

Joscha Bach 1:07:19 ... and, uh, make sense of them. It's not that I believe that, uh, it's your job in life to get to a particular kind of state, and then you get a high score.

Lex Fridman 1:07:26 Mm-hmm. Or maybe you do. I, I, I think you're really against this high scoring thing. I kind of like it.

Joscha Bach 1:07:33 Yeah, you're probably very competitive, and I'm not. (laughs)

Lex Fridman 1:07:35 No, not competitive. Like role-playing games like Skyrim, it's not competitive. There's a, there's a nice thing, there's a nice feeling where your experience points go up. You're not competing against anybody, but it's the world saying, "You're on the right track. Here's a point."

Joscha Bach 1:07:51 That's the game saying it. It's the game economy. And I found when I was playing games and was getting addicted to these systems, then I would g- get into the game and hack it. So, I get control over the scoring system and would no longer be subject to it.

Lex Fridman 1:08:05 So you're not, n- no longer playing, you're trying to hack it.

Joscha Bach 1:08:09 I don't want to be addicted to anything.

Lex Fridman 1:08:11 Hmm.

Joscha Bach 1:08:11 I want to be in charge. I want to have agency over what I do.

Lex Fridman 1:08:14 A- addiction is the loss of control for you?

Joscha Bach 1:08:16 Yes. Addiction means that you're doing something compulsively.

Lex Fridman 1:08:20 Mm-hmm.

Joscha Bach 1:08:20 And the opposite of free will is not determinism, it's compulsion.

Lex Fridman 1:08:26 You don't want to lose yourself in the, in the addiction to something nice? Addiction to love, to the, to the pleasant feelings we humans experience?

Joscha Bach 1:08:35 No, I find this, uh, gets old.

Lex Fridman 1:08:37 Hmm.

Joscha Bach 1:08:38 It is... You... I don't want to have the best possible emotions, I want to have the most appropriate emotions. I don't want to have the best possible experience, I want to have an adequate experience that is serving my goals, the stuff that I find meaningful in this world.

Lex Fridman 1:08:54 From the biggest questions of consciousness, let's explore the pragmatic, the projections of those big ideas into our current world. Uh, what do you think about LLMs, the recent rapid development of large language models, of the AI world, of generative AI? Um, how much of the hype is deserved, and how much is not?

Lex Fridman 1:09:20 And people should definitely follow your Twitter, because you explore these questions in, um, in a beautiful, profound, and hilarious way at times.

Joscha Bach 1:09:28 No, don't follow my Twitter. I already have too many followers.

Lex Fridman 1:09:31 Yeah.

Joscha Bach 1:09:31 At some point, it's going to be unpleasant. I notice that, uh, a lot of people feel that, uh, it's totally okay to punch up. And, uh, it's a very weird, uh, notion that you feel that you haven't changed, but your account has grown, and suddenly you have a lot of people who casually abuse you.

Lex Fridman 1:09:51 Mm-hmm.

Joscha Bach 1:09:51 And, uh, I don't like that, that I have to block more than before, and I- I don't like this overall vibe shift. And right now, it's still somewhat okay, so, uh, pretty much okay, so I can go to a place where people work on stuff that I'm interested in, and there's a good chance that a few people in the room know me, so there's no awkwardness. But, um, when I get to a point where random strangers feel that they have to have an opinion about me one way or the other, I don't think I would like that.

Lex Fridman 1:10:19 And random strangers, because of your kind of el- uh, in their mind, elevated position.

Joscha Bach 1:10:26 Yes. So basically, uh, whenever you, uh, are in any way prominent or some kind of celebrity, uh, random strangers will have to have an opinion about you.

Lex Fridman 1:10:36 Yeah, and they kind of forget that you're human, too.

Joscha Bach 1:10:39 I mean, you notice this thing yourself that, uh, the more popular you get, the higher the pressure becomes. Uh, the more winds are blowing in your direction from all sides. And, um, it's stressful, right? And it does have a little bit of upside, but it also has a lot of downside.

Lex Fridman 1:10:55 I think it has a lot of upside, at least for me currently, at least p- perhaps because of the podcast.

Joscha Bach 1:11:00 Mm-hmm.

Lex Fridman 1:11:01 Because most people are really good, and people come up to me, and they have love in their eyes. And over a stretch of like 30 seconds, you can hug it out, and you can just exchange a few words, and you, you reinvigorate your love for humanity.

Joscha Bach 1:11:16 Mm-hmm.

Lex Fridman 1:11:17 So, that's an upside-

Joscha Bach 1:11:18 Yes.

Lex Fridman 1:11:19 ... for a loner. I'm a loner.

Joscha Bach 1:11:21 (laughs)

Lex Fridman 1:11:22 Because... (laughs) Because otherwise, you have to do a lot of work to find such humans. And here, you're like thrust into the full humanity, the goodness of humanity, for the most part. Of course, maybe it gets worse as you become more prominent. I hope not. This is pretty awesome.

Joscha Bach 1:11:42 I have a couple handful of very close friends, and I don't have enough time for them and attention for them as it is, and I find this very, very regrettable.

Lex Fridman 1:11:50 Yeah.

Joscha Bach 1:11:50 And then there are so many awesome, interesting people that-

Lex Fridman 1:11:53 Yeah.

Joscha Bach 1:11:53 ... I keep meeting, and I would like to integrate them in my life, but I, uh, just don't know how, because, um... Well, there's only so much time and attention. And the older I get, the harder it is to bond with new people in a deep way.

Lex Fridman 1:12:06 Yeah. But can you enjoy... I mean, there's a picture of you, I think, with Roger Penrose and Eric Weinstein and a few others that are interesting figures. Can't you just enjoy random interesting humans-

Joscha Bach 1:12:18 Very much.

Lex Fridman 1:12:18 ... for a short amount of time?

Joscha Bach 1:12:20 I'm also... I, I like these people, and I... What I like is intellectual stimulation, and I'm very grateful that I'm getting it.

Lex Fridman 1:12:26 Can you not be melancholy? Or maybe I'm projecting. I hate goodbyes. Can we just not hate goodbyes and just enjoy the hello, take it in, take in a person, take in their ideas, and then move on through life?

Joscha Bach 1:12:40 I think it's totally okay to be sad about goodbyes, because that indicates that there was something that you're going to miss.

Lex Fridman 1:12:46 Mm-hmm. Yeah, but it's painful. Maybe that's one of the reasons I'm an introvert, is I hate goodbyes.

Joscha Bach 1:12:57 (laughs) But you have to say goodbye before you say hello again.

Lex Fridman 1:13:02 I know. But it, uh, uh, that, that experience of loss, that mini-loss, maybe that's a little death. Maybe, I don't know. I think this melancholy feeling is just the other side of love, and I think they go hand in hand, and it's a beautiful thing. And I'm just being romantic about it at the moment.

Joscha Bach 1:13:26 And I'm not no stranger to melancholy, and sometimes it's difficult to bear to be alive. Sometimes it's just painful to exist.

Lex Fridman 1:13:33 Mm-hmm. But that, there, there's beauty in that pain too. That's what melancholy feeling is. It's not negative. Like, melancholy doesn't have to be negative.

Joscha Bach 1:13:43 Can also kill you.

Lex Fridman 1:13:44 Well, we all die eventually. Now, (laughs) a- as we got to this topic, the actual question was about what are your thoughts on our, about the development, the recent development of large language models with ChatGPT.

Joscha Bach 1:13:59 Indeed.

Lex Fridman 1:13:59 There's a lotta hype. Is some of the hype justified? Which is, which isn't? What're your thoughts, high level?

Joscha Bach 1:14:09 I find that, um, large language models do help with coding, right? So it's an extremely useful application that is for a lot of people, um, taking Stack Overflow out of their life, in exchange for something that is more efficient. I feel that, um, ChatGPT is like an intern that I have to micromanage.

Lex Fridman 1:14:28 Hmm.

Joscha Bach 1:14:28 I've have been working with people in the past who were less capable than ChatGPT. And, uh, I'm not saying this because I hate people. Uh, they, personally as human beings, there was something present that was not there in ChatGPT, which was why I was covering for them. But, uh, uh, ChatGPT is, uh, has an interesting ability. It does give people super powers.

Lex Fridman 1:14:52 Mm-hmm.

Joscha Bach 1:14:52 And the people who feel threatened by them are the prompt completers. They are the people who do what ChatGPT is doing right now. So if you are not creative, if you don't build your own thoughts, if you don't have actual plans in the world, and your only, uh, job is to summarize emails and to expand, uh, in simple intentions into emails again, then, uh, ChatGPT might look like a threat. But, uh, I believe that it is a very beneficial technology that allows us to create more interesting stuff and make the world more beautiful and fascinating if we find to, uh, build it into our life in the right ways. So I'm quite, uh, fascinated by these large language models, but I also think that they are n- by no means the final development, and it's interesting to see how this development progresses. One thing that, uh, the out of the box vanilla language models have as a limitation is that they have still some limited coherence and ability to construct complexity.

Lex Fridman 1:15:53 Mm-hmm.

Joscha Bach 1:15:53 And, um, even though they exceed human abilities to do what, uh, what they can do one shot, um, typically when you write a text with a language model, or using it, or when you write code for the language model, you, you, it's not one shot, because there are going to be bugs in your program, and design errors, and compiler errors, and so on. And your language model can help you to fix those things.

Lex Fridman 1:16:15 Mm-hmm.

Joscha Bach 1:16:15 But this process is out of the box not automated yet.

Lex Fridman 1:16:18 Mm-hmm.

Joscha Bach 1:16:18 So there is a management process that also needs to be done. And there are some interesting developments, baby AGI and so on, that are trying to automate this man- uh, this management process as well. And I suspect that soon we are going to see a bunch of cognitive architectures where every module is, in some sense, a language model or something equivalent. And between the language models, we exchange suitable data structures, not, uh, English.

Lex Fridman 1:16:44 Mm-hmm.

Joscha Bach 1:16:45 And, uh, produce compound behavior of this whole thing.

Lex Fridman 1:16:49 So do some of the, quote-unquote, "prompt engineering" for you.

Joscha Bach 1:16:52 Yeah.

Lex Fridman 1:16:52 They create these, yeah, these kind of cognitive architectures that do the prompt engineering-

Joscha Bach 1:16:56 Yes.

Lex Fridman 1:16:56 ... and you're just doing the high, high level-

Joscha Bach 1:16:59 Yeah.

Lex Fridman 1:16:59 ... meta-prompt engineering.

Joscha Bach 1:17:00 Mm-hmm. There are limitations in a language model alone. I feel that part of my mind works similarly to a language model, which means I can, um, yell into it, a prompt, and it's going to give me a creative response.

Lex Fridman 1:17:14 Yes.

Joscha Bach 1:17:14 But I have to do something with those response first. I have to take it as a generative artifact that may or may not be true.

Lex Fridman 1:17:22 Mm-hmm.

Joscha Bach 1:17:22 It's can, it's usually a confabulation. It's just an idea. And then I take this idea and, uh, modify it. I might build a new prompt that, uh, is, uh, stepping off this idea and develops it to the next level, or it, uh, put it into something larger, or I might try to prove whether it's true or make an experiment. And this is what the language models right now are not doing yet.

Lex Fridman 1:17:44 Mm-hmm.

Joscha Bach 1:17:45 But there's also no technical reason for why they shouldn't be able to do this. So the way to make a language model coherent is probably not to use, uh, reinforcement learning until it only gives you one possible answer that is, uh, linking to its source data, but it's using this as a component in a larger system that can also be built by the language model or, uh, is enabled by a language model structured components, um, or using different technologies. I suspect that language models will be an important stepping stone in developing different types of systems. And one thing that is s- severely missing in the form, uh, of language models that we have today is real time world coupling, right? It's difficult to, um, do perception with a language model and motor control with a language model. I- instead you would need to have different type of, of thing that is, uh, working with it. Also, the language model is a little obscuring what its actual functionality is.

Lex Fridman 1:18:43 Mm-hmm.

Joscha Bach 1:18:44 Some people associate the structure of the neural network of the language model with a nervous system, and I think that's the wrong intuition. Um, the neural networks are unlike nervous system. They are more like, um, 100 step functions that-... use, um, differentiable linear algebra to approximate the ph- uh, correlation between adjacent brain states. It's basically a function that moves the step, uh, system from one representational state to the next representational state.

Lex Fridman 1:19:13 Yeah.

Joscha Bach 1:19:13 And so it's, uh, if you try to map this into a metaphor that is closer to our brain, imagine that you would take a- a language model or a model like DALL-E, uh, that you use, for instance, this image guide- image guided diffusion to approximate a camera image and use the activation state of the neural network to interpret the camera image, which, in principle, I think, will be possible very soon.

Lex Fridman 1:19:35 Mm-hmm.

Joscha Bach 1:19:35 Um, you do this periodically and, uh, now you look at these patterns, how... when this thing interacts with the world periodically, uh, look like as, uh, in time. And these time slices, they are somewhat equivalent to the activation state of the brain at a given moment.

Lex Fridman 1:19:52 How is the actual brain different? Just the asynchronous craziness?

Joscha Bach 1:19:59 Uh, for me, it's fascinating that they are so vastly different, and yet, in some circumstances, produce somewhat similar behavior.

Lex Fridman 1:20:06 Right.

Joscha Bach 1:20:07 And, uh, the brain is, first of all, different, because it's a self-organizing system where the individual cell is an agent that is communicating with the other agents around it and is always trying to find some solution, and all the structure that, uh, pops up is emergent structure. So one way in which you could try to look at this is that individual neurons probably need to get a reward so they become trainable, which means they have to have inputs that are not affecting the metabolism of the cell directly, but they are messages, semantic messages that tell the cell whether it has done good or bad and in which direction it should shift its behavior. And once- once you have such an input, neurons become trainable, and you can train them to perform computations by exchanging messages with other neurons. And parts of the signals that they are exchanging and parts of the computation that they are performing are control messages that perform management tasks for other neurons and other cells. I also suspect that, uh, the brain does not stop at the boundary of neurons to other cells, but there are many adjacent cells will be involved intimately in the functionality of the brain and, uh, will be instrumental in distributing rewards and, um, in managing its functionality.

Lex Fridman 1:21:20 It's, uh, fascinating to think about what those characteristics of the brain enable you to do-

Joscha Bach 1:21:25 Yes.

Lex Fridman 1:21:26 ... that language models cannot do.

Joscha Bach 1:21:27 So first of all, it's, there's a different loss function at work when we learn.

Lex Fridman 1:21:31 Yeah.

Joscha Bach 1:21:31 And, uh, to me, it's fascinating that you can build a system that looks at 800 million pictures and, uh, captions and correlates them-

Lex Fridman 1:21:39 Mm-hmm.

Joscha Bach 1:21:40 ... because I don't think that the human nervous system could do this. For us, the world is only learnable because the adjacent frames are related and we can afford to discard most of that information during learning. We basically take only in stuff that makes us more coherent, not less coherent. And our neural networks are willing to look at data that is not making the neural network coherent at first-

Lex Fridman 1:22:02 Mm-hmm.

Joscha Bach 1:22:02 ... but only in the long run. By doing lots and lots statistics, eventually patterns become visible and emerge. And, um, our mind seems to be focused on finding the patterns as early as possible.

Lex Fridman 1:22:13 Yeah, so filtering early on.

Joscha Bach 1:22:15 Yes.

Lex Fridman 1:22:15 Not later.

Joscha Bach 1:22:16 Yes. It's a slightly different paradigm and it leads-

Lex Fridman 1:22:18 Yeah.

Joscha Bach 1:22:18 ... to much faster convergence, so we only need to look at a tiny fraction of the data to become coherent. And of course, we- we do not have the same richness as, uh, our trained models. We are, we will not incorporate the entirety of text in the internet and be able to refer to it and have all this knowledge available and being able to confabulate over it. Instead, we have a much, much smaller part of it that is more deliberately built. And to me, it would be fascinating to think about how to build such systems. It's not obvious that they would necessarily be more efficient than us on a digital substrate, but I suspect that they might. So I suspect that the actual AGI that, uh, is going to be more interesting is going to use slightly different algorithmic paradigms or sometimes massively different algorithmic paradigms than the current generation of, uh, transformer-based learning systems.

Lex Fridman 1:23:08 Do you think it might be used in just a bunch of language models like this? Do you think the current transformer-based large language models will take us to AGI?

Joscha Bach 1:23:20 Uh, my main issue is, uh, I think that they're quite ugly and, uh, brutalist.

Lex Fridman 1:23:25 Which- which- which... brutalist-

Joscha Bach 1:23:26 Yes.

Lex Fridman 1:23:26 ... is that what you said?

Joscha Bach 1:23:27 Yes. They are basically brute forcing the problem of thought. And, uh, by training