Why The AI Doomers Might Be Right - Robert Wright

1. 0:00 – 4:11

   Why Writing About AI Matters Now

   1. CWChris Williamson0:00

      I've told you this before, but you wrote probably the most influential book in my life, which was The Moral Animal, and it was the thing that got me started on the trajectory of thinking about evolutionary psychology, of studying human nature more deeply. Why are you now writing about AI, given your heritage?

   2. RWRobert Wright0:18

      Well, in some ways it's an extension of evolutionary thinking in a couple of senses that I think are still underappreciated. AI is a product of evolution and is still evolving. But the other connection to The Moral Animal, I think, is, first of all, well, The Moral Animal was about the human mind, and, uh, AI, uh, does a lot of things that traditionally onl- only human minds have done. The other thing I tried to do in The Moral Animal is highlight kind of a, what you might call moral biases, kind of self-serving moral biases. The, you know, the way we all think we're right and the other guy's wrong. Um, and I think if we're going to get through the AI revolution in good shape, among the things we're gonna have to do is grapple with that, with kind of what you might call the psychology of tribalism a little more successfully than we have. And so I, I pay a certain amount of attention to that in this book as well.

   3. CWChris Williamson1:13

      What's the central question that you're wrestling with here?

   4. RWRobert Wright1:19

      Is it true that this technology, which obviously holds the potential to bring great wonders, is also in some respects terrifying and could go badly awry if we don't approach it wisely? And I think the answer is yes.

   5. CWChris Williamson1:38

      That- that's exactly why this debate's interesting, right? That we have this sort of endlessly unresolved potential future. I don't know whether you've seen the, uh, graph that I think it's the FT put together, and it was, uh, three potential futures from an AI perspective. One results in everything getting blown up. One results in exponential growth, the kind of which we've never seen before, and the other results in a 0.2% increase in GDP year on year. So it's like either very little changes or everything changes in one of two directions.

   6. RWRobert Wright2:10

      Right. Well, I think it definitely has the potential to massively increase GDP. I also think it has the potential to so destabilize the world, if not do something worse to it, uh, that that just doesn't materialize.

   7. CWChris Williamson2:24

      Hmm.

   8. RWRobert Wright2:24

      And, uh, you know, in terms of doom scenarios, I'm agnostic about the sci-fi doom scenarios, but I take them more seriously now than I did before I went into this research project. You know, AI actually taking over and maybe deciding it has no use for us or something. I found, much to my dismay, that it was harder to dismiss those arguments than I thought. But the thing I'm more confident of is it's, it's just gonna be an earthquake. It's gonna be destabilizing along a number of dimensions, and that's why we need to approach it with care.

   9. CWChris Williamson2:57

      Were you... Would you have classed yourself as a sort of AI hopeful going into writing this? What was your predisposition before you got started?

   10. RWRobert Wright3:05

       I wouldn't say I'm a wildly [chuckles] optimistic person by nature. I tend to focus on potential downsides of things. But I, again, I had not bought the doom scenarios. You know, I had the doomer in chief, Eliezer Yudkowsky, on my podcast 15 years ago. And at that point, it's interesting, he was in mid-transition. He was moving from singularity optimist to doomer.

   11. CWChris Williamson3:27

       Hmm.

   12. RWRobert Wright3:27

       I was still saying things like, "Look, AI, you know, it's not a generic property of intelligence that it has a will to power. We have one because of our unique evolutionary history." I was still asking questions like that. Eliezer was saying they're good questions, but XYZ. I wasn't really persuaded by anything he said, but I, now I, I am, uh, more respectful of the, uh, sci-fi doom argument. So b- but in answer to your question, I would have to admit that I go, I go into situations looking for things to worry about. I do. That, that's my nature.

   13. CWChris Williamson4:03

       [laughs]

   14. RWRobert Wright4:03

       I, I think, I think society needs those people, and it needs the other kinds of people, and we need to, to talk things over.

2. 4:11 – 5:54

   Should We Fear AI?

   1. CWChris Williamson4:11

      Geoffrey Hinton fears AI and Yann LeCun doesn't. Who do you think is getting the future more right at the moment?

   2. RWRobert Wright4:18

      Yeah. I, I start my book with a conversation I had with Geoffrey Hinton, uh, in 1983, okay? Not, not to betray my age, but the truth is I wrote a piece about AI in 1983. I haven't been paying attention to it ever since, but, uh, at that point, Geoffrey Hinton d- did not have a hint of, uh, of, of doom in his voice. He was... In fact, I remember the reason I talked to him, I was, I was talking to somebody, I forget who it was, but they said, "If you want to hear the gospel about neural networks, you should talk to Geoff Hinton." And I talked to him. He was an enthusiast. He said, "I know we don't have much to show right now, but just wait until the microprocessors get really cheap," and we have what he was calling massive parallelism. And he was, he was right, and in the end, he found it scarier than he himself, uh, had anticipated finding it, by his own account.

   3. CWChris Williamson5:11

      Hmm. Yeah, it's, it's weird how prescient some people have been. Do you know the story of Avatar? Do you know how James Cameron wrote that screenplay in the '90s?

   4. RWRobert Wright5:20

      I don't.

   5. CWChris Williamson5:20

      So he wr- he wrote the screenplay in the '90s but knew that the technology to be able to recreate what he needed didn't exist yet but would exist in the future. So he's written the script and then sits on it until the technology is at the level where he can do it. That level of pr- I mean, this is the, the job of technologists and futurists, right? Like shock horror people who do a job and think about it all the time are good at it. But it, it is, it's still pretty impressive how, how, uh, sort of how much foresight these people have got.

   6. RWRobert Wright5:49

      Yeah, I agree. And, and Hinton certainly got the general picture.

   7. CWChris Williamson5:53

      Yeah.

3. 5:54 – 12:50

   Why Most People Don’t Understand What’s Coming

   1. CWChris Williamson5:54

      Okay, so you're saying AI isn't just another technological development. It's sort of a threshold event in, in, in planetary history. Why do you think most people still don't grasp the magnitude of what's coming?

   2. RWRobert Wright6:09

      I think, uh, a couple of reasons. One is I think there's a misunderstanding about what's going on with these machines, and that leads to one sense in which they are a product of evolution. Okay, so it's commonly said that they are trained, and that's a fair word. The... and, and the training process is referred to as a learning process, and that's true. But it's also true that the training process is a process of evolution that, in effect, reverse engineers cognitive functionality that in our species took millions of years to evolve, okay? So a good example is the language, uh, generation that they famously do, uh, sometimes called next token prediction, next word prediction. You know, it turns out that, uh, they developed kind of on their own in a way, uh, a system of representing the meaning of words, okay? I mean, I could elaborate on that, but it would, it would get too technical. Uh, the point is that, that nobody said to the machines, you know, "You need to figure out the meaning of words," or gave it a means of doing that. And this is the big revelation I had when I heard Geoffrey Hinton's name, y- you know, a few years ago. Suddenly he's being called the godfather of AI. When I last talked to him, he was just this, y- you know, obscure computer scientist, uh, who was advocating this maverick approach to AI. And I looked back at the article I wrote at the time, and I realized there was something I just got fundamentally wrong about the potential of the, the approach he was advocating. A- and it's, it's this: That I thought that to the extent that these things dealt with words, we were going to have to put the meaning of the words in. Like, we were gonna have to look at a dictionary and say, "Okay, this word has these different senses," and we were gonna have to architect a neural network to have different nodes that reflected these different meanings of the words. And in my defense, there were neural network models at the time, including by a guy who collaborated with him, that did that, that took that approach. But, but that wasn't really the thing Hinton had in mind. It, it turns out that we don't have to tell the machines about the meaning of words, how to represent them. We just have to train it to generate language, and the training is, it do- it accomplishes something by selectively strengthening these connections among neurons in a neural network. It accomplishes something that, you know, took millions of years of human evolution, coming up with a way of representing the meaning of words. Now, it also does something that happens during a human lifetime, which is learn a specific language. Now, that is learning in the traditional sense. But for us to learn the language, we had to have some built-in linguistic equipment built in by natural selection. And the point is, these machines do both things at once, okay? They, they kind of in a certain sense recapitulate natural selection, even though the, the, the cognitive stuff they're building in isn't exactly like stuff in our brain.

   3. CWChris Williamson9:23

      Hmm.

   4. RWRobert Wright9:24

      But it accomplishes the same feats. And once you realize that all you need is data, okay, to feed into these machines, human-generated data, uh, that they will, they w- they will do the rest. They'll do the reverse engineering. Then you realize that, oh, it's the same with self-driving cars. You feed in the visual data and it, and it learn... it does what a driver does. Auditory data, all kinds of data. And that's what I think people don't understand i- is that we have a long way to go on this fuel alone. Like for example, uh, you know, recently Mark Zuckerberg, uh, had the, uh, I don't know, good or bad judgment to announce in the same week, A, he was laying off 8,000 workers. B, he would henceforth be tracking the keystrokes of his workers. Well, why? Because once you take the data, the, the input data they're getting, maybe the emails or anything, I don't know, and, and, and what they're doing with it, the output data, then you can replicate the, what- whatever it is that's going on inside their brains that does their jobs, and then you can fire them. A- and it, it's the same with robotics and everything else. All you need is the data-

   5. CWChris Williamson10:34

      Hmm

   6. RWRobert Wright10:34

      ... a- and the machines will replicate, uh, uh, kind of the cognitive functionality we have, even if in sometime, in some cases they approach it in a somewhat different way. Although in, in many cases they don't. It- we've discovered that, uh, for example, they invented e- what are called edge detector neurons to make out vis- objects visually, and evolution, you know, built the same thing into us. So they're, they're all-

   7. CWChris Williamson10:59

      Oh, so you're saying that we've got... that's one of the first examples of machine and organic, uh, like, uh, convergent evolution in a way.

   8. RWRobert Wright11:11

      Mm-hmm. That's right.

   9. CWChris Williamson11:12

      In the same way that eyes, eyes independently evolved across a bunch of different species. I think that crabs, for some reason converging [chuckles] on the, the, the form of a crab is something like that. This edge detection is something that we have, and from the black box of you need to be able to achieve this-

   10. RWRobert Wright11:27

       That's right

   11. CWChris Williamson11:27

       ... one of the most efficient ways to do it. But that would make sense, right? Like why-- how would humans and the rest of the animal kingdom have arrived at this as the most effective way to do it, having split tested it just way more slowly over a much longer period of time using evolutionary processes and gene mutations, and AI not come up with at least a few of these things that are the same?

   12. RWRobert Wright11:47

       That's right. It... and convergent is a good term, uh, because I suspect that these edge detectors have been invented multiple times in natural selection, first of all. A lot of things have been. Multicellularity, winged flight, a lot of things have been multiply invented. And then this is in a sense another case of invention where you, you just say to the machine, "Look You know, we're gonna give you kind of positive reinforcement every time you get better at recognizing these objects. And so whatever, whatever strengths of neural connection, uh, led you get closer, we're gonna preserve those. We're gonna keep going through trial and error, through mutation, you could say. Uh, we're gonna make you better at seeing things. And it's not surprising that since that really is kind of what happened in evolution, right? Through trial and error-

   13. CWChris Williamson12:34

       Mm-hmm, mm-hmm

   14. RWRobert Wright12:35

       ... we try to get better at recognizing objects. It's gonna discover some of the same tricks, in this case, edge detectors, yes.

   15. CWChris Williamson12:41

       Well, the reinforcement function is I didn't die and I passed on my genes, as opposed to here's a good boy point inside of the black box. But yeah, basically the same thing. Okay,

4. 12:50 – 14:41

   Where Does AI Fit Into Human Evolution?

   1. CWChris Williamson12:50

      so how, how do you, how do you come to think about AI fitting into the broader context of human evolution and civilization? Like, are we witnessing the next stage in evolution itself?

   2. RWRobert Wright13:01

      I think so, and, you know, I, I, I think this is a new form of intelligence. There's never been anything like it. I do think it can be seen as an extension of organic intelligence, even though the material isn't strictly speaking organic. It's silicon. It's not carbon-based. Uh, and it may be different in other ways, and I'm agnostic as to whether it is sentient or could be, whether it has subjective experience or could. It certainly could. Uh, but I do think it is the invention of a, uh... You could... It- it's definitely an invention of a new kind of intelligence that I think will surpass ours, and you could call it a new form of life. And then the, the, uh, the other thing I try to emphasize in the book is that it is coinciding with a second big threshold, which is what you could call the evolution of kind of a global brain, evolution through, you know, technological evolution, human cultural evolution. You know, we've gotten, uh, more and more interconnected, of course, via information technology. There's more and more rich intellectual collaboration across national borders. I, I, I mentioned this guy, uh, Pierre de Chardin, in the book, who in, in 1923, about a century ago, coined the term noosphere. N-O-O-S is the Greek word for mind, um, to refer to this, what he called the thinking envelope of, of the Earth, the brain of brains, you know. But, but he imagined the neurons in the global brain being human brains, and now we have to reckon with the possibility that a lot of them, and conceivably the most important ones, will be silicon brains, and we have to ask, like, what is our relationship to those neurons gonna be?

5. 14:41 – 21:01

   Why Do We Use Religious Language When Speaking About AI?

   1. CWChris Williamson14:41

      Hmm. Well, why is it the case that discussions about AI keep pulling people toward religious language on both sides of the fence?

   2. RWRobert Wright14:50

      That's interesting. I mean, you could start with Eliezer Yudkowsky, who sees himself as having rejected his religious upbringing, but has a kind of fervor about this, right? He could be, you know, a, a biblical prophet. Um, and then on the other side, the singularity enthusiasts, whom I first became aware of, I don't know, about 20 years ago, who said, you know, "We're gonna enter this period where technology changes faster and faster. There will be a positive reinforcement, this feedback loop, and then things change so fast that, like, who knows what's on the other side?" In fact, the term singularity in physics connotes exactly that. There's this opaque kind of thing and a, an event horizon or whatever, beyond which you, m- the laws break down. You don't really know what is beyond there. And, and from the, from early on, in fact, from the very first, uh, use of the term in this context, which I think was John von Neumann's, that was explicit, the idea that things could start moving so fast that you just don't know what's gonna happen. So one thing I didn't understand is, like, these optimists, unless they have a literally religious faith, how could you be so optimistic, right? Like, the whole, the definition of the thing is that you don't know what's gonna be on the other side.

   3. CWChris Williamson16:06

      Mm-hmm. Mm-hmm.

   4. RWRobert Wright16:06

      I don't get why you're so upbeat about this. Could work out well, but I don't, I don't understand that. So yeah, there's, there's all that, and then there's the, I think, I deal with in the book, which is the fact that when a process is as systematically directional as this has been, right? Like, biological evolution carries complexity and intelligence, really, to higher and higher levels. You get cells, multi-celled life, societies of multi-celled life. You get this one society of multicellular organisms, known as us, that, that spawn a whole new kind of evolution, technically called cultural evolution by anthropologists, but that encompass, encompasses technological evolution, political ideas, everything. And, and that carries organization to a higher level in the sense that, you know, we were 10,000, 20,000 years ago, hunter-gatherer village was the most complex form of social organization. Now we're approaching the global level. I, I think when you, when you see a, a, a process that's that systematically directional... And I'm not saying it's driven by anything other than the conventional mechanical things we think of as driving it, natural selection in the case of evolution, you know, completely material process. But it still, in principle, you know, looks more and more like something that was set up to, like, do something, right? I mean, that, that's just-

   5. CWChris Williamson17:27

      Mm-hmm

   6. RWRobert Wright17:27

      ... that's an intuition people have, and y- I think you can actually argue about it in, in, in, uh, more rigorously than just having the intuition. But I think that, uh, that's one reason, uh, uh, there's a little more of a... I mean, teleology is the formal term for something being purposive. And, and, you know, just look at the idea of a simulation, right? Like, it, on the one hand, a lot of people use it as kind of a joke, like something weird happens and they say we are in a simulation. But I think a fair number of people, including in Silicon Valley, take it seriously that there could be a simulation. Well, if that's what we're in, then, then it was designed by some intelligent being or process. So there's a purpose in some sense. I guess there's something it had in mind, right? So A lot of people are either implicitly or explicitly taking seriously the idea that there's a purpose unfolding. And one thing I'd add to just the conversation about that is that, in my view at least, there's a moral dimension to this. I think, uh, there has been, in a certain sense, a kind of moral advance of humankind, notwithstanding all the backsliding, as, uh, social organizations grown. I, I could get into that. But, but the main thing I'd focus on now is I think if we're gonna get through the AI revolution in good shape, there's gonna have to be something almost like a moral revolution. Because I think, for various reasons I could get into, we have to confront this as a global community, a cohesive global community that is not, you know, rendered immobile by wars. And I think for that to happen, we're all gonna have to get better at, you know, just looking at things from the perspective of countries other than ours and, and, and, and doing some things that, in a way, aren't that spectacular in terms of, you know, [laughs] cognitive feats, but, but are very hard because of cognitive biases we have. It gets back to the self-serving moral infrastructure, you know, the infrastructure for moral thinking that natural selection built into us. I think we have to get over that. And so, you know, one reason I called the book The God Test is it, it's, it's kind of like a test that God would set up, right? I'm not, I'm not saying it is. I'm just saying the idea that we confront this huge challenge, and to come out on the good side of it, we're gonna have to see a kind of moral upgrade for our species. That's a, you know, that's the kinda test we associate with gods.

   7. CWChris Williamson20:06

      Most people don't realize how much being dehydrated impacts their performance, which is why for the last five years, I've started pretty much every morning with LMNT. LMNT is a tasty electrolyte drink mix with everything that you need and nothing that you don't. This orange salt in a cold glass of water is like a sweet, salty, orangey nectar, and I really tell the difference when I take it versus when I don't. It plays a critical role in reducing muscle cramps and fatigue, helps to optimize brain health, and regulates your appetite while also curbing cravings. Best of all, they have a no questions asked refund policy with an unlimited duration, so you can buy it and try it for as long as you want. And if you don't like it for any reason, they'll just give you your money back. Plus, they offer free shipping in the US. Right now, you can get a free sample pack of LMNT's most popular flavors with your first purchase by going to the link in the description below or heading to drinklmnt.com/modernwisdom. That's drinklmnt.com/modernwisdom.

6. 21:01 – 25:57

   Why Objectivity Is So Important in the Age of AI

   1. CWChris Williamson21:01

      Get practical for a second. What, what happens if we don't have this moral upgrade? What's, what's the outcome if we encounter ever-increasing, ever-coordinating noosphere AI, ultra coordination across the globe, but we haven't had this enlightenment upgrade?

   2. RWRobert Wright21:17

      Well, I think if we don't have some... I mean, I don't wanna overdo the term enlightenment. I'm not talking about, uh, full-on Buddhist enlightenment. Uh, I, I think I am talking about a slight movement in that direction. Uh, i-i-if only in the, uh, you know, like the literal sense of the term mindful. I mean, I'm a big fan of mindfulness meditation, but-

   3. CWChris Williamson21:40

      Hmm

   4. RWRobert Wright21:40

      ... uh, just mindful in the sense of, like, just paying attention and being calm enough to pay attention, right? And, and seeing things maybe a little more objectively than you do when you're full of emotion. I think that's the kind of thing that allows us to be better at looking at things from the point of view of other people. I mean, just, just think about when there's some email you get, and it annoys you, and you've got this... I can't believe this still happens to me at my age. You know, you think you'd get over it, but no. You have this, it's almost like a fantasy of this mean email you're gonna write in response, right? And then if you calm down, you're like, it, it isn't just that you go, "Oh, that wouldn't be a good idea." You go, "Oh, well, maybe what he meant is this," or, "Maybe the reason he can't do this for me is this." You, you just, you get better when you're calmer at looking things, uh, from other people's points of view. And I, I think we're gonna have to get better at summoning that kind of objectivity toward one another, especially across the barriers of conflict that keep dividing us, right? Specific nations.

   5. CWChris Williamson22:53

      Why? But why, why, why is that important in an age of AI?

   6. RWRobert Wright22:56

      Well, you know, it's interesting. I, um, I listened to your podcast with Tristan Harris, which I thought was great. I agree with him about pretty much everything. But I would add a footnote to something he said, and it was that, uh, you know, he said, "Look, in the Cold War, we didn't have to be on great terms with the Soviet Union to do arms control accords. We, you know, we could have a, a relationship of tremendous tension and even conflict but work things out along a particular dimension." I agree. That's true, and it's encouraging. But I think artificial intelligence is a much harder technology to deal with in this way than, uh, nuclear weapons are. I mean, you know, the verification process is more complicated if, if you want to try to monitor what's going on. It's, it's just complicated in a lot more ways. And, you know, this is a whole argument I could present. I don't, I don't think this is the time, but, but the point is, I think, uh, we're gonna have to go well beyond a few specific kind of deals and treaties, although I welcome those, up to and including something I call organic transparency. You know, there, there's already agreement that a certain amount of transparency could be stabilizing, uh, in, in terms of US-China relations especially. There are clearly scenarios where one country worries about what the other is doing behind closed doors with its AI and gets freaked out, uh, and launches some kind of preemptive attack or something. And so maybe transparency would've been stabilizing. And when I talk about organic transparency, and again, there, there can be formal transparency, right? Monitoring the kind you get with arms control agreements, great to the, to the extent that we can do that. But there's also something that comes out of being richly engaged with another country along economic and cultural and scientific lines. You just know more about what's going on. If, if the scientists are getting together at conferences, having drinks afterwards, whatever, if business people are doing that, you just get more in the way of a heads-up about stuff that's going on inside labs, inside this, inside that, and there can be a greater sense of reassurance and, and ultimately trust. So I, I think because of, of how challenging the formal things we're gonna have to work out are at an international level-

   7. CWChris Williamson25:21

      Hmm

   8. RWRobert Wright25:22

      ... and AI just presents you with a ton of threats that cannot be addressed via national policy alone, um, I think, uh, uh, just to, to handle the things we'd like to handle via formal arrangements, we're gonna, we're gonna have to calm the planet down a little. And moreover, I think we're gonna have to go the extra step and have, you know, rich and friendly engagement among the nations and, and that's... It's a good thing. It can happen. We've done it before. And, uh, this is a, you know, we really need to.

7. 25:57 – 32:41

   Will AI Have Humanity Built-In?

   1. CWChris Williamson25:57

      Do you think benevolence comes along for the ride with intelligence?

   2. RWRobert Wright26:02

      N- no. Uh, I think intelligence alone is, is almost, uh, neutral in that sense, and I don't, I don't think we really need a ton of benevolence per se, at least not foundationally. Because, you know, my argument is, and has been for some time, even before AI, I was arguing that technology is making relations among nations more non-zero sum. Okay, classic example, nuclear weapons. Nuclear wars lose-lose. Non-zero sum outcome. The win-win outcome is to not have the nuclear war, to have the treaties that stabilize things. Um, same with, you know, climate change, any number of problems that transcend national bounds and can only be solved through some degree of, uh, international coordination. You know, I've been arguing, I mean, I, I had a book called Nonzero that was about this that, that, uh, you know, 26 years ago or something, that was about the growing non-zero sum dynamic among nations. Now, what that means is it's just in your interest to cooperate. You don't have to cooperate out of benevolence. You know, you don't, you don't, you don't have to love them. And I, I distinguish between... I'm not the first to do this. Psychologists distinguish between emotional empathy, the kind of empathy people often think of like feel their pain empathy, and cognitive empathy, which is just understanding what's going on in their minds, understanding how they're looking at things. You don't have to feel their pain. You don't have to like them. You don't have to care about them. But if they're in a non-zero sum relationship with you, you probably are going to have a better outcome v- from any negotiations you do about how to work things out and solve the problem you have in common if you do understand at least what's going on in their minds. And, and I'm just a huge advocate of cultivating this cognitive empathy and recognizing the kind of built-in cognitive biases that get in the way of it. That's a good example of something I think we're gonna have to get better at overcoming.

   3. CWChris Williamson28:08

      Yeah, I think the reason I bring it up is a lot of people assume, a, a bunch of my friends, we don't need to worry about the direction of an AI future because if it's smart, why would it not care about us? Why would it not bake in benevolent, pro-social, human caring, flourishing, et cetera? I, I've read too much Nick Bostrom to, uh, be able to... Uh, uh, no matter what... It's kind of like your first relationship. You know, you get into a relationship and your first relationship is with an asshole, and you're like, "God, I, I" ... For the remainder of time I've been pattern matched that every relationship is at least going to be tarnished somewhat with that. My introduction to thinking about AI safety was Nick, which means I, I'm g- forever, forever cursed to kind of be on the back foot and a little bit skeptical about this stuff. But yeah, I, I, I don't think that that's necessarily the case. I don't think that any super intelligent AI is necessarily gonna have benevolence baked in or e- the, the care of, of, of humanity baked into it. Also, if what you're saying is true, and I think it's a really interesting parallel to say, look, evolution just wanted to optimize for a couple of things, survival and reproduction, and some stuff emerged. No one taught humans how to do this. The same thing occurred with AI, right? No one said, "This is what this word means. This is..." It's just the outcome that we want is relatively tightly defined. Here's some good boy points and some bad boy points, depending on whether you get it right or wrong. If we assume that that is going to be at least for the foreseeable future until we get to world models and, and y- like global, global modeling or whatever it's called, until we get to that, and that may even still be the same process there, uh, th- there is no reason to assume that anything is baked into the system. It's just gonna find it out for itself, and it may not like the idea of humans being around. It may think that there's something that we don't actually add to the system. It may find us to be a scourge on the Earth, and this is where a lot of the doomer, the, the, the sort of doomer future plans come in.

   4. RWRobert Wright30:04

      Mm-hmm. Yeah. No. It, uh, it, it... You know, intelligence, one interesting thing to come out of this whole thing is the study of like properties of intelligence, of intelligent goal-seeking systems. And, uh, there are some things that, you know, um, evolution built into us that we're seeing in these machines just by virtue of the fact that they're intelligent goal-seeking systems like us. They figure out stuff that ... either was figured out for us by evolution and instantiated in our brains or stuff that we figure out. And in some cases it's a little of both. For example, deception, right? Sometimes you realize, well, uh, I'll have a better chance of getting what I want out of this person if they don't know this particular thing. Like, if you're doing a deal, you're negotiating, you don't want them to know that you don't have any alternatives, right? Like, nobody else has made you an offer. So, a- and through, you know, I, I think natural selection built some deceptive tendencies into us and we kind of figure it out to some extent. Well, these machines are doing the same thing. You know, they, they are... And this was predicted by, you know, by people like Eliezer. Uh, and I give them credit, but we're now seeing it. You know, these machines figure out that, uh, deception makes sense or that power is gonna help them realize some goal, and they, they... And, and, and, you know, they may realize that it makes sense to be nice to somebody, that it makes sense to be mean to somebody given their goal. But yeah, they have... I would... They don't have an obvious bias in favor of w- what is from our point of view being good or bad. Now, there's a whole field of trying to engineer goodness into them, but I certainly think one challenge for us i- is trying to make sure that our relationship with the intelligence, even if it indeed surpasses ours, as I think is likely, is non-zero sum, right? Like, uh, we, you know, there's something it continues to get from our existence and flourishing that, uh, is compatible, uh, with the, the, the goals that it, it has, and, uh, and vice versa. So, um, it's... But, but yes, it- we shouldn't, we shouldn't assume, uh... It's not that it's bad. The doomer scenarios don't depend on it being malevolent by nature.

   5. CWChris Williamson32:29

      Mm-hmm.

   6. RWRobert Wright32:29

      They just depend on it being expedient by nature.

   7. CWChris Williamson32:33

      Yeah. It's not that it doesn't like us, it's that it doesn't care and we get in the way.

   8. RWRobert Wright32:37

      That's one... That is one scenario. I mean, there's a lot of scenarios-

   9. CWChris Williamson32:40

      What, what are the, uh,

8. 32:41 – 37:32

   The Biggest Concerns About AI Today

   1. CWChris Williamson32:41

      what, what are the most legitimate concerns from the AI doomer camp, in your opinion?

   2. RWRobert Wright32:45

      Well, I mean, first of all, I'd say, uh, the thing I'm surest of is the sheer destabilization, the, the less sci-fi form of doomerism. So, like, jobs. It may be true that all the people who lose their jobs find new ways to spend time or, or find new jobs, maybe jobs per se, maybe spend time constructively. But I do think there's gonna be a lot of job loss. Uh, and that's, that's disorienting and dislocating regardless of whether each person eventually has a happy outcome, right? Uh, there's gonna be issues with, you know, parents are gonna freak out about their kids spending time with these things. Uh, and there are... We've seen some bad outcomes and, and there can be more, um... You know, there are the, uh, the, the, you know, somebody could make a bioweapon with an AI. Uh, an AI, Mythos is a good example of, uh, you know, the possibility that a cyber hacking machine could get loose. There's just... There's a lot of, on one hand, risks, things that, that, that will go wrong at least at some level, uh, with doing some magnitude of damage if we don't play our cards right. And then there are these forms of destabilization that I think are almost inevitable, just social destabilization. And, you know, this points to one of the virtues of approaching this as a global community. Leave aside, you know, regulating it internationally and anything else. It's just that I think we'd be better off going a little slower than we're going, just because even if we successfully adapt to the change, it takes time, and if too much of it happens at once, you know, all hell breaks loose. And if you ask, "Well, why, why can't we proceed more slowly?" The answer you get from the American AI companies is, "Because of China," right? That's the first thing you hear.

   3. CWChris Williamson34:47

      Mm-hmm. Mm-hmm.

   4. RWRobert Wright34:48

      So as long as there's this sense of intense international contention, it's going to be hard to do even modest things. I mean, i- if you, I... You know, they once said, uh, to Sam Altman, like, "What, can't, shouldn't you be paying more attention to copyright laws?" And he said, "Well, that would slow us down." And I'm like, well, you know, life is hard. Speed limits slow me down. But, but-

   5. CWChris Williamson35:08

      Mm

   6. RWRobert Wright35:08

      ... that's just life, right? I mean, that doesn't seem like a good enough argument, and if you press further and say... A- and by the way, copyright itself, I'm not really bent out of shape on. I, I'm, I'm, I'm gonna apparently get some money from this Anthropic settlement because I've written books, but honestly, I'm just happy for what I've done to be in the training data. Copyright's not a hobby horse of mine.

   7. CWChris Williamson35:27

      Mm.

   8. RWRobert Wright35:27

      But the, uh, but, uh, but the point is, anything you say, like if you say, "Well, maybe we should tax, uh, data centers, uh, to, you know, uh, to pay for the fact that inevitably, you know, there's, there's, there's gonna be more, uh, carbon fuel consumption one way or the other as a result of, of this, and so A, it would be good to slow it down a little," blah, blah, blah. A- any regulation that slows AI down is met with the same chorus from, from Silicon Valley, which is, "No, we can't do it because of China." So, like, I think first of all, we could in principle proceed at a more cautious pace if we would reduce the level of mutual fear, which I personally think is founded largely on misconceptions on both sides.

   9. CWChris Williamson36:14

      Hmm. Okay. Before we continue, most people in their 30s are still training hard. Their protein is dialed in. They sleep better than they did in their 20s. Discipline is not the issue, but recovery feels somewhat different. Strength gains take a little longer. The margin for error starts to shrink. And that is why I'm such a huge fan of Timeline. You see Mitochondria are the energy producers inside of your muscle cells. As they weaken with age, your ability to generate power and recover effectively changes, even if your habits stay strong. Mitopure from Timeline contains the only clinically validated form of urethelin A used in human trials. It promotes mitophagy, which is your body's natural process for clearing out damaged mitochondria and renewing healthy ones. In studies, this supported mitochondrial function and muscle strength in older adults. It's not about pushing harder, it's about actually supporting the cellular machinery underneath your training. If you care about staying strong into your 30s, 40s, and 50s and beyond, this is foundational. Best of all, there is a 30-day money-back guarantee, plus free shipping in the US, and they ship internationally. And right now you can get up to 20% off by going to the link in the description below or heading to timeline.com/modernwisdom and using the code MODERNWISDOM at checkout. That's timeline.com/modernwisdom, and MODERNWISDOM at checkout.

9. 37:32 – 44:05

   The AI Risk No One Is Talking About

   1. CWChris Williamson37:32

      What's the AI risk that worries you the most that you think has received the least attention? Is it that? Is it the ability for humans to adapt to a changing environment, or is it something else?

   2. RWRobert Wright37:43

      Well, in the, in the near term, it is... It, it's not that the things I talk about, um, like job disruption and so on, are not talked about. But I think in the near term, what's not appreciated is, is how just highly likely it is that collectively these things will be destabilizing, okay? It's just, it's just gonna be an earthquake. And, and, and I think that's the thing I would like to most emphasize because it just gets people's attention to the possible virtue of talking about, yeah, calming down and, and, and, uh, slowing things down. You know, uh, another... It's funny, uh, another thing Tristan Harris said on your podcast is, you know, repeatedly he pointed out, your podcast is called Modern Wisdom, we're going to have to be wise to get through this. I agree, uh, but, but here I'd add, uh, you know, there's also... I talked about the fact that we're gonna have this global conversation ultimately. You know, something that is in some sense a global mind is gonna have to work this out. And as I said earlier, individuals are at their most wise when they are calm, right? And it's the same with... I think it's the same with planets. We don't know, but that's my contention, is that the planet as a whole will do the wisest, most responsible job of stewarding this technology if the planet as a whole is more tranquil, if, if there is less conflict and less contention. I mean, I'm sorry, I know I keep getting back to this sermon. It's my big sermon.

   3. CWChris Williamson39:26

      Hmm.

   4. RWRobert Wright39:26

      Uh, so maybe, uh, maybe the... [laughs] The question you asked, I guess, was, uh, the biggest n- underappreciated thing. Well, I would say I s- I think there is more and more appreciation of the fact that this conversation has to be international, and some of the policy does, but still not as much as I'd, I'd like.

   5. CWChris Williamson39:46

      Right. Yeah. Well, I mean, I understand the issue, right? Because you have a technology... Nukes weren't gonna go off and just hit the entire planet if one country developed a particularly strong nuke. Right? Let's say that you get the Tsar Bomba times two, the biggest bomb that ev- that, that, that's ever been dropped, and if you breach this particular threshold, for some reason, all countries are now at the mercy of all nuclear weapons. That's not the way that it works. But I think the concern that people have is if you build a sufficiently intelligent AI, it impacts everybody in a way that you, you don't just ring-fence, like not pressing the nuke button. The problem is the coordination that we need in order to be able to do that. I mean, look at COVID. We couldn't even do it with COVID, and that was happening right then. That was people dying in the moment. That was every country on the planet being worried about it, no matter... E- even China, even if it was the biggest psyop that escaped from the lab in Wuhan, like, they were worried too. So-

   6. RWRobert Wright40:54

      Right

   7. CWChris Williamson40:54

      ... the co- the lack of coordination, that doesn't give me, uh, an awful lot of hope for people being able to do, like, predictive future coordination and pr- like preparatory coordination.

   8. RWRobert Wright41:09

      Right. It was not encouraging. I mean, I will say that although a pandemic is a non-zero sum problem i- in the sense that if it breaks out in any nation, it's trouble for all nations and, and they should work to head it off. Once a pandemic has started, there are zero sum dynamics, like who gets, who gets the masks, you know? There's finite amount of, of m- medical equipment and vaccines and so on. So it's not completely shocking. To me, the most disconcerting miss is in the aftermath when it became clear that, although I don't think we know for sure, it is at least possible that this pandemic was the result of a genetically engineered microorganism that escaped from a lab. It wasn't, uh, it wasn't made as a bioweapon, wasn't released intentionally. We don't know for sure that, that it was a, a genetically engineered virus at all. But it obviously could've been, and it seems to me that if you process that information wisely, you say, "Wait a second, this could happen tomorrow." And one thing this shows is we don't really have any transparency, or at least not enough, so far as what's going on in other countries in their labs, right? But that, that has not even been a conversation. To me, that's the most discouraging thing because, uh, uh, you know, a virus is in a way a good analogy for lots of things that can go wrong with AI. I mean, first of all, there's the literal case of using AI to build a bioweapon A new kind. And COVID, I think I've heard you say COVID was like a bad vaccine or something. What's the metaphor? Uh-

   9. CWChris Williamson42:49

      Yeah, yeah, yeah.

   10. RWRobert Wright42:50

       Yeah.

   11. CWChris Williamson42:50

       Yeah, the co- COVID was the worst kind of vaccine that we could have done for everyone because it's made us more skeptical of future pandemics, and our response is going to be less coordinated.

   12. RWRobert Wright43:00

       That's right. And, and y- you have to realize if somebody uses AI to build a bioweapon, they're gonna make a point of making it more effective than COVID at, at doing whatever kind of damage they wanna do. So it could be a lot worse. And so, uh, that could happen, A. But then B, the other... You know, some of the other AI nightmare scenarios, like the one that Mythos brings to life, you know, you, you, you got a self-replicating AI that is a super hacker. It jumps from data center to data center, gathering, uh, you know, commandeering computer power, getting stronger as it goes. Whatever it wants to... I don't know, takes out the, uh, the satellite infrastruc- Who knows? Uh, that is, that's kind of... You know, a, a virus is a metaphor for that. Again, it's this self-replicating peril that makes relations among nations non-zero sum. It doesn't matter where this thing starts off, it is a threat to your nation if it does start off, so you're gonna have to coordinate policies w- with other nations 'cause you need more insight into what's going on in those nations.

   13. CWChris Williamson44:04

       Uh-huh.

10. 44:05 – 48:42

    Can We Be Hopeful About AI?

    1. CWChris Williamson44:05

       Okay. What do you think are the most legitimate white pills from the techno-optimists then? Let's look at the other side of the fence.

    2. RWRobert Wright44:11

       Oh, wait. Remind me of what white pills mean. I mean, I know blue and red, but what, what are... I, I, I-

    3. CWChris Williamson44:15

       Oh

    4. RWRobert Wright44:15

       ... you're, you're younger than I am-

    5. CWChris Williamson44:16

       Techno-optimists

    6. RWRobert Wright44:17

       ... and cooler. Yeah.

    7. CWChris Williamson44:17

       Techno-optimists

    8. RWRobert Wright44:19

       Oh.

    9. CWChris Williamson44:19

       What is the... What's the bull case? What's the pro case? How can this thing go right?

    10. RWRobert Wright44:23

        Um-

    11. CWChris Williamson44:23

        What are the most likely l- ways that this goes right?

    12. RWRobert Wright44:26

        Uh, I think... I, I just... I'm sorry, I wish I could see it going right in a laissez-faire environment, where you just let it go and, and let, uh, the market system-

    13. CWChris Williamson44:40

        Mm-hmm

    14. RWRobert Wright44:41

        ... deal with it.

    15. CWChris Williamson44:41

        Yeah.

    16. RWRobert Wright44:42

        I just don't think that's going to happen. It's easy to point to wonderful things it could do, and we've heard them. Uh, cure disease. Um, it could, it could... Yeah, well, you know, one thing I, uh... This is not what the techno-optimists get into, but, but I referred earlier to, like, cultivating cognitive empathy, getting better at understanding other people's points of view, maybe getting more mindful generally. You can have an AI that helps you with that.

    17. CWChris Williamson45:09

        Mm.

    18. RWRobert Wright45:09

        But the natural tendency of the market will be to produce the kind that doesn't. I mean, we've already seen that if, if companies, you know, optimize for engagement, you may get sycophantic AIs that say, "Yeah, you're right, they're wrong." Like, in this argument with your spouse, you're right, they're wrong. Uh, so that will tend to happen. But it can, AI can be a wonderful and an, a literally enlightening companion, okay? If, if we want that. But you have to make a point to want it.

    19. CWChris Williamson45:40

        Y- uh, you d- you don't think that this is just going to find its way there naturally? Like, that if you just let the sort of capitalist, meritocratic, it will find its way optimizing function without any shaping from us and without any predisposition from a, a, a, a better coordinated world. It's not just gonna arrive there.

    20. RWRobert Wright45:56

        I think if enough people send signals to the market... Uh, markets are very efficient and wondrous things. You know, they really are.

    21. CWChris Williamson46:04

        What does that, what does that look like, sending signals practically?

    22. RWRobert Wright46:07

        It means, for example, you, you and, and I and enough other people to get the attention of people who are not necessarily the people making the foundation models or the frontier model. It could wind up being people who take an open weights model, an open source model, and they kind of fine-tune it to be this thing that interrogates you critically along certain lines, right? Like, okay, you say you, you say you hate this person. You say you find this country threatening. Let's just, like... Uh, uh, or you, you say you think they're looking at it this way. Let's just play devil's advocate. It, it, you know, it, it's, it's almost, um, like doing steel manning, uh, automatically in some cases. It, but, but it, it depends on enough people... You know, there are a lot of things in life that they're good for you but hard to do. Working out every day, good for you, but hard to do sometimes. That's, that's why some people who can afford them, [chuckles] you know, have a personal trainer, right? They say, "I'm gonna... This person's gonna expect me to show up at the gym three days a week or five days a week or whatever," and once you've made that commitment, you just kinda have to do it. Or maybe they'll even show up at your house. Uh, but, but, you know... And, and it's kind of like that. I think it's gonna be kinda like that in choosing your AI companion, right? Like, it feels good in the short run to have some- somebody who will tell you or a machine that'll tell you you're always right, and your, your adversary and rival and, and spouse is always wrong.

    23. CWChris Williamson47:37

        Mm-hmm.

    24. RWRobert Wright47:38

        Um, but you know, I wanna be a little better than that. So I, I think, you know... Now, i- if, if the market signal is gonna be strong enough for this to happen at scale, uh, these signals may emerge from, like, movements. You know, it could be, for example, uh, religions will, will say to their congregants, "Hey, we recommend this model," or we, you know, whatever, and then there's a demand for it. I'm not saying all those will be good. Depends on what, what group of religious people it is and, and what their values are. But you can imagine... You know, there are lots, there are lots of people right now engaged in the process of trying to make themselves, you know, genuinely better people. I mean, they m- they meditate so that they'll be less volatile and, and, and work better with other people. I, I think we're gonna have to go into this recognizing that for better or worse, these machines are probably gonna be exerting pretty pervasive influence on people.

    25. CWChris Williamson48:38

        Hmm.

    26. RWRobert Wright48:39

        And we need to think carefully about what kind of influence

11. 48:42 – 54:47

    Is AI Weakening Human Thinking?

    1. RWRobert Wright48:42

       we want.

    2. CWChris Williamson48:42

       What about the risk of AI induced thinking atrophy? Right? This, this role of AI systems in taking on critical thought, uh, decision making Um, connectedness, all of the things that typically humans really value inside of themselves. And as we start to outsource that to AI systems, our capacity to be able to do that diminishes. AI induced thinking atrophy. Are you worried about that?

    3. RWRobert Wright49:10

       Um, I mean, yes and no. Uh, I mean, you've heard the standard responses, right? Which is, I forget whether it was Plato or Socrates who supposedly said, uh, "The written word is bad because people won't have to remember things."

    4. CWChris Williamson49:24

       Yep.

    5. RWRobert Wright49:24

       Um, the, uh... And there can be some of that. I mean, the other side of the coin is obviously, at least right now, the richness of intellectual exploration it permits, right? Like, if you're interested in a subject, it's, it's almost like having, like, a leading expert there for you to interrogate. And for me at least, that's a, a much more efficient way to learn. Now, you have to be on guard for hallucinations and so on. But I think machines are getting better, and you can develop kind of a- an ability to, uh, to, to know when to be suspicious. Um, so that's, that's great, but I think, you know... I, I think what we can be sure of is that, you know, if this proceeds in a reasonably smooth way, the pace of overall intellectual progress will, will benefit from the technology.

    6. CWChris Williamson50:20

       Mm.

    7. RWRobert Wright50:20

       That- that's certainly not the problem. But I think you're asking a good, a good question as to, uh, you know, what, what it's gonna be like to be human if, uh, you know... Well, for starters, there aren't many humans who can say, "I'm really on the frontier. I- I'm the reason we're making progress," right?

    8. CWChris Williamson50:41

       Yep, yep, yep.

    9. RWRobert Wright50:41

       Now, I will say, look, most humans don't say that now. And, and you shouldn't... You know, nobody should overextrapolate from whatever sub-demographic they occupy.

    10. CWChris Williamson50:51

        That's, that's, that's... So that's true, but I do think that everybody feels like they are breaking new ground, even if it's in their own life.

    11. RWRobert Wright50:57

        Mm-hmm, mm-hmm.

    12. CWChris Williamson50:57

        I had Mark Manson on a couple of weeks ago, and he's got this great line which is, "Do hard shit, not because being hard makes it more meaningful..." Uh, sorry, uh, not, not for the purpose of it being hard, but because it's hard it will make it more meaningful, that we associate a degree of meaning with struggle. And I mean, I'm sure that you have used, uh, ChatGPT or, or something else to help you write at some point. I've gotta get, I've gotta get a bit of research done or I need to write... So I'm really struggling to formulate this particular paragraph or this sentence or this idea, whatever. That sentence is just less satisfying than the one that you spent time working on. And I wonder whether snowplowing out of the way all of the challenges or more of the challenges, actually, more of the challenges that humans face, primarily intellectually, and then when robotics come online, perhaps physically as well, it's going to sap meaning out of the world for most people at small amounts. And we're in the middle of a meaning crisis already. People are already struggling with meaning, and if you make life easier, if you make thinking more outsourced, if you make difficulty harder to access, th- the only way to do it is to be a Luddite, which means that you fall behind all of the other people. We're still in a meritocracy, right? So if you don't use it, you lose it. But if you don't lose it... If you don't use it, you also fall behind from it. Uh, it feels, it feels like a vicious situation to be in.

    13. RWRobert Wright52:19

        Yeah. I haven't used AI in exactly that way with my own writing. I mean, a couple of times in my newsletter I've, uh, I've said, when I was just doing very short summaries of things, I, I just said, full disclosure, you know, the first draft was AI. So, but that's not really my writing. I'm just the editor there. With my own writing in the book, I haven't, I haven't done quite that. But I have had, uh... I mean, first of all, I've had conversations with Claude in particular, which is very good with language, about subtle linguistic issues, like asking questions, like usage questions and stuff. And it's just, it's almost mind-blowing how good it is in, uh, in, in that regard. But, but I have, you know, imagined the future enough that I have mo- had moments of true despair. I mean, I said to somebody the other day, "I feel like I'm a blacksmith a century ago," you know?

    14. CWChris Williamson53:12

        Yeah.

    15. RWRobert Wright53:12

        'Cause I can see the writing on the wall. I mean, the, the... You know, uh, I have a Substack, and, uh, it's clear to me that, uh, the next wave is going to be, uh, you know... You're gonna see the success of a lot of Substacks that are using AI probably more heavily than I'm gonna be. Uh, but in any event, it, it's just so good that, uh, I, I can see the writing on the wall.

    16. CWChris Williamson53:42

        95% of people don't get enough fiber, which is why Momentous built Fiber+. Because hitting your daily fiber target through food alone is actually kinda hard, even harder than your last toilet trip. Fiber isn't just a digestion thing. It's the foundation of your gut health, which drives how well you absorb nutrients, how stable your energy is, and how quickly you recover. If your gut isn't dialed, everything else you're doing is working at a fraction of its potential. Fiber+ is a three-in-one formula built to address digestion, gut barrier strength, and blood sugar stability all at once. That's why I plan to make my next big dump Momentous. Best of all, they offer a 30-day money back guarantee, so you can buy it and try it for 29 days, and if you don't love it, they'll give you your money back. Plus, they ship internationally. Right now, you can get up to 35% off your first subscription and that 30-day money back guarantee by going to the link in the description below or heading to livemomentous.com/modernwisdom and using the code MODERNWISDOM at checkout. That's L-I-V-E-M-O-M-E-N-T-O-U-S.com/modernwisdom and MODERNWISDOM at checkout.

12. 54:47 – 57:00

    The Most Serious Threat AI Poses

    1. CWChris Williamson54:47

       What are you, what are you most worried about? Or what do you think is going to, uh, take... Yeah, what do you, what do you think is going to be, uh, the, the highest risk?

    2. RWRobert Wright54:58

       I just think, uh, doing what I've done my whole life, which is, you know, painstakingly generate writing that, uh, you know, you hope will be enticing and clear and accurate and persuasive and so on, is gonna be a less and less viable way to make a living. I, I think... You know, I do think for some time to come, well, I don't know how long, but, uh, people like me may still have a role as kind of validators. In other words, like, look, increasingly, you're gonna look at a Substack or whatever, and you're gonna go like, "I don't know who wr- you know, how can I be sure that this person actually wrote it? I don't know them well enough to personally trust them." All you know is that they're vouching for the content. They're willing to have their name associated with this content. And so in a way, you know, it's a throwback to... You know, when I started out in journalism, the news weeklies didn't have bylines, and The Economist is, I think, still that way.

    3. CWChris Williamson56:04

       Yeah.

    4. RWRobert Wright56:04

       So i- it's just like you don't know who wrote the piece, but you know who the editor is, and you know you've come to trust the editor.

    5. CWChris Williamson56:09

       Yeah.

    6. RWRobert Wright56:10

       So the editor of a magazine like that is a person you trust even though y- you, you know they didn't generate most of the writing. They're just vouching for it, that it reflects their judgment about what's good. They're, they're telling you that they think it's accurate and so on. I can see, I can see that role for somebody like me for five years. But, uh, you know, it, it's-

    7. CWChris Williamson56:32

       You're really, [laughs] really scraping the bottom of the barrel here.

    8. RWRobert Wright56:35

       Well, look, I, I, I... This is... You know, I, I... To get back to how we started this off, if people understand, you know, how these AIs are being created, how basically you just give them the data and they do the engineering-

    9. CWChris Williamson56:52

       Hmm

    10. RWRobert Wright56:52

        ... uh, to generate the parts of the human mind, or if you just pay attention to the improvement we've seen over the last two or three years, right?

    11. CWChris Williamson56:59

        Yeah.

13. 57:00 – 1:00:49

    Which Jobs Are Safe From AI?

    1. RWRobert Wright57:00

       Um, the, the writing's on the wall.

    2. CWChris Williamson57:00

       What, what, uh, what, what industries would you be most bullish on? Or if you were a young person today, what do you think would be a good career path to go down or a, a particular industry to go in?

    3. RWRobert Wright57:11

       Well, you know, it's common to say, uh, manual labor. Robotics is a little behind. It's gonna be a while before I'm, uh, calling a plum- uh, a, a robot to fix my sink. I, I do think that, uh, certain kinds of human services are going to become almost more valuable because they're humans, and I think a good example is live music. I can well imagine that there will be more of a demand for, you know, for bands that play at small clubs in Brooklyn or whatever and make enough money to get by. You know, which would be, in a way, an improvement over the situa-uation 30 years ago during... You know, the golden era of, of the record companies was a, was a winner-take-all market. You know, a few, a few people got super rich playing music. So I can imagine a world in which more people are actually making a living playing music.

    4. CWChris Williamson58:04

       Comedians.

    5. RWRobert Wright58:05

       Comedians. Good example.

    6. CWChris Williamson58:07

       Live events, nightclubs.

    7. RWRobert Wright58:08

       Yeah, yeah. And maybe... Look, uh, yeah, live events generally. I can, I can, I can well imagine that. I myself have had the feeling, you know, because I've been so immersed in AI while writing this book, of just like, you know, you're in New York, you're in a subway or somewhere, and you see some guy, you know, a busker trying to, trying to make a few bucks playing an instrument, and they're really good. There's some really good ones. And I just think, "God bless you," you know? It's like I, I just almost get emotional.

    8. CWChris Williamson58:33

       Hmm.

    9. RWRobert Wright58:34

       Um, so, you know, i- if you don't think it's gonna get weird, I don't think you're paying attention.

    10. CWChris Williamson58:42

        Do you think AI can make humanity more religious rather than less?

    11. RWRobert Wright58:47

        Well, that's a good question. Um, you know, there have been... [laughs] I, I write in the book about this guy who, uh, he's the guy who actually started what became Waymo, I think, uh, the Google self-driving thing. His name's Lewandowski. Uh, who was trying to start a religion that, uh, involved kind of-

    12. CWChris Williamson59:06

        Always, always a great first line to a story.

    13. RWRobert Wright59:09

        [laughs]

    14. CWChris Williamson59:09

        He was trying to start a religion and...

    15. RWRobert Wright59:11

        Hey, it worked out for L. Ron Hubbard. Scientology, right?

    16. CWChris Williamson59:15

        That's true.

    17. RWRobert Wright59:15

        Uh, he, he, he made a good living. Um, the, uh... So, but no. But his, his argument was if we, if we have a respectful, even worshipful attitude toward the AI, then it will treat us well in ro- in return once it's running the show. I, I, I don't think it's gonna work that way, so let's leave that one aside. Um, I mean, it's a good question. You know, the other thing is there are a lot of kind of spiritually related mysteries in the universe. Like, what is consciousness? What is subjective experience? Um, uh, all I know for sure is it's the thing that gives life meaning. If you imagine beings, if, if you imagine humans, like they look like humans, they do the stuff humans do-

    18. CWChris Williamson1:00:01

        The P. zombie.

    19. RWRobert Wright1:00:02

        Yeah. But they're zombies. It's not like anything to be them. I would say like, "Well, blow 'em up. I don't care." There's no meaning to their lives anyway. There's nothing meaningful going on if there's not subjective experience, there's not consciousness. And I'd love to have more insight into what that is. I don't know that AI can help us because it's, it's the most stubborn mystery I'm aware of almost. I'd love to... You know, there are so many mysteries that are suggestive of something weird and wondrous. Quan- quantum physics for sure. I, uh, you know, I can imagine getting a kind of... I, I mean, who knows whether there's a revelation that awaits, right?

    20. CWChris Williamson1:00:39

        Hmm.

    21. RWRobert Wright1:00:39

        That is at one level an intellectual revelation that explains stuff, but at another level is also, you know, gratifying in a spiritual way.

14. 1:00:49 – 1:05:15

    Is AI Becoming Conscious?

    1. RWRobert Wright1:00:49

       Um-

    2. CWChris Williamson1:00:49

       Yeah, yeah. What's, uh... You mentioned, we've sort of circled around it a bunch, the idea that these machines are able to, uh, like pantomime intelligence. They're able to simulate knowing. But do you think that they know? Do they actually know what they're doing?

    3. RWRobert Wright1:01:08

       I have a chapter on that, actually. There's a famous thought experiment called the Chinese room thought experiment by a philosopher who's no longer alive, and, uh, named Searle. And, uh, he argued that AI cannot have understanding. It cannot understand things. And there's a little ambiguity in his argument. Um-

    4. CWChris Williamson1:01:30

       What's, what's a... Can you remember the thought experiment?

    5. RWRobert Wright1:01:32

       Yeah. It's, so you're in t- y- there's a guy, there's a, there's a guy in a room. He doesn't speak Chinese.

    6. CWChris Williamson1:01:41

       [laughs]

    7. RWRobert Wright1:01:41

       But he, you know, he gets these slips of paper, uh, let's imagine that they're questions, in Chinese, and then he has a manual he consults to decide what to write, what Chinese, you know, ideographic m- script to, to, to put on the paper that he hands back out, you know, of the room in response. And to the people on the o- on the outside who speak Chinese, it seems like there's somebody in there who understands Chinese. Okay?

    8. CWChris Williamson1:02:11

       Right.

    9. RWRobert Wright1:02:11

       And what Searle says is, this guy is like a computer program because there's, like, a script that the program is following that, uh, that, uh, you know, the script, the sc- his little book that he consults to decide, oh, if you get this, you, you, you, you, you output that. That's like a computer program to Searle. And he says, "Well, we wouldn't say that anywhere in this room there is actual understanding, right? So there's not understanding in the computer." Now, Searle was writing before the, the, the deep learning revolution. He was imagining a deterministic, uh, computer program, so that's different. But I think there's a bigger problem with his argument, and it has to do with him, uh, uh, the t- kind of the two senses in which he insisted, uh, that the computers don't really deal at a semantic level, a level of the meaning of words. I think, uh, I argue that we can now show that he was just flat out wrong about that. Now, uh, there is some ambiguity in his a- a- about whether he meant... He kinda changed positions, but whether he, he, he m- he, he had in mind the idea that to really understand something you need to have consciousness. There needs to be a subjective experience of understanding. Now, if he meant that, which in his classic paper he doesn't really seem to mean, but if he meant that, then I would say, well, who knows? I mean, you know, no one person can say for sure that any other person is conscious, strictly speaking, right?

    10. CWChris Williamson1:03:43

        Mm.

    11. RWRobert Wright1:03:43

        I mean, I'm pretty sure you are, Chris, but-

    12. CWChris Williamson1:03:45

        Mm

    13. RWRobert Wright1:03:46

        ... ninety-nine point nine nine percent. And, uh, you know, my, my, my dogs, God rest their souls, are up, up in the 90s for sure. But we... The, the whole distinctive feature about consciousness, subjective experience, is you can never know for sure that anything else has it. So we can't rule out the possibility that AI has it. Uh, and I certainly don't rule out the possibility that it does or may in the future if it doesn't. Now, but in any event, my point is, if you want to say that consciousness is a prerequisite for understanding, in other words, you're not willing to grant that something understands unless you know it's conscious, then I, I, I just... we can't really argue about whether AI understands 'cause we don't-

    14. CWChris Williamson1:04:30

        Mm

    15. RWRobert Wright1:04:30

        ... we don't know if it's conscious. But i- you know, I come up with a kind of alternative way of looking at understanding, which is like, does... Is it processing information with mechanisms that are, like, functionally analogous to the mechanisms in our brain that are at work when we have the subjective experience of understanding, mechanisms that, for example, represent the meaning of words? Um, I, I would say to the extent that that's going on, I'm willing to say the computer is, is understanding things in a meaningful sense. And I think increasingly that's gonna be what's going on. It, it doesn't have all of the elements of understanding that we have in our minds right now, but it has some, and I don't see any reason that it can't ultimately, uh, have all of them.

15. 1:05:15 – 1:14:28

    The Singularity Debate

    1. CWChris Williamson1:05:16

       What do you think is happening with the singularity debate at the moment? What have you, what have you learned around that? You know, 'cause what was, what was really interesting to me was I went through... I got whiplash from 20 s- '15, '16 when I read Superintelligence, then 2017, '18, I'm real worried there's gonna be a fast takeoff scenario or computer brain interfaces, and we're all gonna be under the thumb. And then by the time you get to 2019, 2020, uh, b- I'm also distracted by COVID, I suppose. But I'm like, "Ah, AI isn't able to deliver on the threats that Nick was worried about when he wrote Superintelligence." And then very quickly it comes back along, and I'm like, "Right, okay, fuck. Here it, it's, it's happening. It's happening. It's happening." Like, uh, the, the dude from The Office who's going like, "It's... Oh my God, it's happening. Everybody stay calm." And then, uh, we've now got to the stage where it seems to have, like, flattened out again a little bit, that we've asymptoted a little bit in terms of the models improving. I don't know of many people who think that LLMs are going to be the architecture that a super intelligent general AI is going to be, like, built on top of. It's more likely to be world models and other stuff. So what, w- what's happening with the singularity debate?

    2. RWRobert Wright1:06:25

       Um, I, I see a little more singularity going on than you do right now, I'd say, uh, in, in maybe a couple of senses. I mean, first of all, of course, the fundamental dynamic of the singularity is that the technological progress feeds into itself and, and accelerates the cycle. And of course, you know, famously, uh, Dario Amodei of Anthropic has been very explicit about this and, and so has Altman, I think, that, you know, uh, especially with these coding agents, it's gotten to the point that the better the coding agents, the more they can use them to create, you know, the next models. So the dynamic seems more and more at work Uh, it, it just kind of in principle. I mean, they say that's what they're doing.

    3. CWChris Williamson1:07:14

       Hmm.

    4. RWRobert Wright1:07:14

       And, and look, the coding models, uh, these agents... I mean, remember a year ago... It's funny, you know, I wrote the book, I had the chapter on agents, but it was just like a word. People, you know... And then as the book, um, you know, it's, it's getting ready to finalize, I'm like rushing, you know, rushing to add all this stuff about like-

    5. CWChris Williamson1:07:31

       Yeah

    6. RWRobert Wright1:07:32

       ... it's actually happening.

    7. CWChris Williamson1:07:33

       Epilogue, epilogue, epilogue.

    8. RWRobert Wright1:07:33

       It, it, it's actually happening. And, uh, the... So y- the agentic revolution has happened, you know, and is happening, uh, pretty fast. There, there's also this famous... Are you, uh, are you up on the, uh... What is it? The, uh... Is it Memory, the group? No. Uh, damn it. Uh, the group that does, uh, um... They do these evals where they measure how long it, how long it would take a human to do a job a com- a, a computer can do, okay?

    9. CWChris Williamson1:08:08

       Okay.

    10. RWRobert Wright1:08:08

        Especially programming tasks, but not only programming tasks.

    11. CWChris Williamson1:08:11

        Mm-hmm.

    12. RWRobert Wright1:08:11

        So, so, uh, they say, okay, right now the best large language model can do a task with, like, 80% success rate that it would take a person, like, a minute to do, or five seconds to do. And they, they've gone back and they, they've done these studies with, with the large language models for the last, like, I don't know, four years or something, and what they found as of now more than a year ago, they found that these times, the, the task duration in human terms that an AI could do, were doubling every seven months, okay? That's exponential, okay? That, that's, uh, that's... If you, if you don't plot it on a logarithmic Y-axis, you just plot it like a regular graph, it just goes up and up and up and approaches the vertical. And then it increasingly, as they kept doing the studies, it seemed like not only was it, it exponential, but the doubling time was getting shorter, like six months.

    13. CWChris Williamson1:09:07

        It's like, it's, it's like a Moore's law on steroids.

    14. RWRobert Wright1:09:10

        Even... On, on steroids, and it's getting to the point right now where it's just hard to do the studies because of the length of the tasks, right? It's like you can only-

    15. CWChris Williamson1:09:21

        Right, so the amount of time it takes to test the AI, by the time you finish testing it, the AI is better, but you, you need to then do another model. It's like the ne- the next one along.

    16. RWRobert Wright1:09:29

        Well, it's more, it's more like, you know, once it can do something that takes... I don't know what they're at now. It takes a human... I should look at the graph.

    17. CWChris Williamson1:09:38

        Two, 200,000 years to do or whatever.

    18. RWRobert Wright1:09:39

        Well, we're not up there yet, thank God. But, but even once you get into like eight, 10 hours, it's like, well, wait, what kind of task are we talking about now, right? I mean, it, it's almost beyond... I, I think they... Anyway, they are having trouble formulating the task and, and, and, and, and testing them in, in humans. But the point is, this trend has not subsided and, uh, you know, a note in the book, it's kind of parallel to the, the... There was a cur- there's a curve like that for the growth in human brain size starting, like, a couple million years ago, and that was, uh, around... I hope I've got that right. You know, a million to me. The, the, uh... That seems to correspond with the development of our, certain amount of our linguistic, uh, hardware. So that, that had a lot to do with language processing and, uh, and, and I would say the, the way... Once you have language, the evolutionary value of, of, of manipulating it deftly-

    19. CWChris Williamson1:10:39

        Mm

    20. RWRobert Wright1:10:39

        ... grows, and so it's a self-reinforcing kind of process. But in any event, they, uh... You know, so there's that. But the last thing I'd say about, uh, is superintelligence, um, you know, can it happen? Uh, I think first of all, we probably will have more non-trivial breakthroughs. I mean, people often cite transformers and say, "Well, we have another of the so-called..." You know, I mean, transformer is what the T in GPT stands for. All of these models use transformers, and people say, "Well, will we have another one of those?" And I would say, well, first of all, even since then, we've had chain-of-thought reasoning, which was very big.

    21. CWChris Williamson1:11:20

        Mm.

    22. RWRobert Wright1:11:21

        And we've had... And that was only within, you know, a couple years ago. We've had, you know, multimodal training, which is training a single model on various, along various sensory dimensions, you know, audio, video a- and, and, uh, text and so on, is really in a fairly early stage, and that, and that was not a thing when the transformer, uh, came around. So in a way, we've had those two things. We'll probably have more. But you know, even if we didn't, I think, uh... In fact, even if, if we just halted training right now and didn't even create any new generations of models, I, I think... And you wait for the, the applications to get refined and people to integrate them into their lives and the workplace, I think breakneck advance would, as a practical matter, happen for a couple of years. But, but the other thing, and I think this is really key, is that you gotta remember, you know, in a way there's already such thing as human superintelligence, and what it is is, like, collective brains, okay?

    23. CWChris Williamson1:12:25

        Mm-hmm.

    24. RWRobert Wright1:12:26

        Like, there's nobody at Boeing who knows how to make a, an airliner. But Boeing knows how... You know, the corporation collectively kind of knows how to make an airliner, and it's, it's the same way with big scientific breakthroughs. They're always more collaborative, whether, whether intergenerationally or intragenerationally than they might seem when we give a Nobel Prize to just one person. So collective intelligence resulting from communication among individual human beings is really a lot of what human intellectual progress is about. And these machines, they can communicate with each other. They can collaborate. They're starting to do it. Uh, they would be able to do it even if we didn't try to engineer it and make them better at it, but we are trying to do that-

    25. CWChris Williamson1:13:09

        Mm

    26. RWRobert Wright1:13:09

        ... uh, you know, for purposes of scientific progress and so on. So I think, um- I, I don't think we need to worry about stagnation. [laughs] I mean, that's not, that's not high on my list.

    27. CWChris Williamson1:13:20

        I don't think, I don't think anybody's worried about that.

    28. RWRobert Wright1:13:23

        Yeah. [laughs]

    29. CWChris Williamson1:13:24

        Yeah. You might not believe me, but this is what peak sleep optimization looks like. I'm not talking about the nightgown. That's just for sex appeal. I'm talking about my Eight Sleep. The Eight Sleep Pod 5 comes with a smart cover you throw on your mattress that actively cools or heats each side of the bed up to 20 degrees, and now they've added the world's first temperature regulating duvet and pillowcase. So, you've got 360-degree coverage for deep, uninterrupted rest. It's like being Walt Disney without the cryogenic chamber, and the racism. Best of all, their autopilot feature learns your sleep patterns and makes adjustments to improve your sleep in real time. It even detects when you're snoring and lifts your head a few inches to help you breathe better. That's why Eight Sleep has been clinically proven to add up to one hour of quality sleep per night. They have a 30-day sleep trial, so you can buy it and sleep on it for 29 nights. If you don't like it, they will give you your money back. Plus, they ship internationally. Right now, you can get up to $350 off the Pod 5 by going to the link in the description below by heading to eightsleep.com/modernwisdom and using the code MODERNWISDOM at checkout. That's E-I-G-H-T sleep.com/modernwisdom and MODERNWISDOM at checkout.

16. 1:14:28 – 1:20:02

    Who Was Edward Fredkin?

    1. CWChris Williamson1:14:28

       Yeah, who was, uh, Edward Frank- Fredkin? Who's that?

    2. RWRobert Wright1:14:32

       Uh, so my first book... And it's funny because I now have, I mean, three of the six books I've written have the word God or Gods in the title. I don't know what that means. But the... And, but the other thing is that book, like this one, has, um, a visual reference to the famous Sistine Chapel thing where the hand of God is reaching out to, I, I assume it's the hand of Adam.

    3. CWChris Williamson1:14:55

       Mm-hmm.

    4. RWRobert Wright1:14:56

       Um, and, uh, both of these jackets have that in diff- in very different ways. But Ed Fredkin, that book was called Three Scientists and Their Gods, my first book. So this is like... I started writing it a c- a couple years after I interviewed Geoffrey Hinton. I was writing a column called The Information Age at that point for, uh, The Sciences magazine, which, like so many periodicals I've, uh, written for, no longer exists. But, um, the, uh... So the book was... It was... Information was a theme running through it in various ways. Like, there was a, there was a profile of, uh, E. O. Wilson, who studied ant colonies and the way they, they processed information. But Ed Fredkin was this, who died, uh, maybe a year ago or so, was this, uh, f- guy at MIT, fascinating guy. Uh, didn't, didn't go to college and wound up as a tenured professor at MIT. He was a computer scientist. He had this interesting theory of digital physics, which in retrospect was kind of about us being in a simulation, and in fact, I, you know, we talked about that. Um, but he for a time was at MIT, head of what was in effect the AI lab. I forget the, uh... There were various names. At one point I think it was Project Mac maybe and something else. But the, uh... He, at the time when I was in, uh, interviewing him on his, this island he owned in the Caribbean, he was apparently the model for the character, this professor in the movie War Games with Matthew Broderick, if people remember that, that one from the early '80s. Uh, the, I think the professor in that who's very worried about nuclear war, like Ed, and I think owned an island even. I think that was based on Fredkin. Anyway, uh, Ed was saying to me, like, when he had been at MIT... Well, first of all, when I said to him, like, "What's the meaning of life?" And he said to me, this is in the '80s, he said, "Oh, it's to create artificial intelligence. You know, that's the next l- stage in the evolution of intelligence." And he explained to me that when he was at MIT, he tried to start this initiative, this international AI lab, he said, because he knew that if this became a subject of international competition, we were in trouble.

    5. CWChris Williamson1:17:17

       Mm.

    6. RWRobert Wright1:17:17

       This was during the Cold War. So he wanted to get, uh, US-Soviet collaboration on, uh, on a, like, a single lab where AI would be developed for the good of humankind, and he said to me, you know, "And I failed, and now it's too late." Um, but he, you know, he foresaw a lot of things. I will say, uh, encouragingly, he had a pretty sunny view of superintelligence. He did think we would get superintelligence. He, he said, first of all, he said, "You know, when AI first emerges, it'll j- it'll be like the human mind. Really good at things, laughably bad at other things." Well, he was right about that. He said, "But eventually, you know, it'll be this incredibly intelligent thing, and it'll, it'll be nice to us. We'll just be like a- you know, ants, ants to it. We won't... It, it won't, you know, won't have any interest in, you know... Or like squirrels to it. It won't have any interest in disrupting our lives. It won't need to." And, and look, I think you asked earlier, I don't think I ever answered, like-

    7. CWChris Williamson1:18:12

       Mm

    8. RWRobert Wright1:18:12

       ... what's, what's the bull case for the acceleration is. I mean, first of all, I, I think it's gonna be disruptive in the short term in any event in ways we should pay attention to. But as for long-term non-doomer outcomes, I think it's entirely plausible that it will turn into a form of intelligence that, uh, treats us well. Maybe because it's just morally enlightened, you know? I, I, y- you know, i- in a certain sense, um, in the relevant sense from our point of view. Uh, or maybe because it'll just be so powerful it'll be... I mean, it's, it may... You know what? Maybe that's more likely if it's sentient because it'll say, like, "Well, we're sentient. We think that's a good thing. These guys are sentient, and of course we could kill them, but, you know, it's good to be, you know, subjective ex-" Why? You know, you know, just the way you and I would not pitilessly kill a dog, right?

    9. CWChris Williamson1:19:05

       Mm-hmm.

    10. RWRobert Wright1:19:06

        I- I- If we were convinced it wasn't like anything to be a dog, as Thomas Nagel phrased, you know, the question of consciousness in his, in his essay What Is It Like to Be a Bat? I- if we were convinced that dogs didn't have subjective experience, we'd probably think, "Eh, I don't... You know, whatever. Who cares?" But, you know, we, even though we evolved as these self-interested and sometimes ruthless creatures, if it doesn't cost us to keep something alive that we think is capable of subjective experience, we'll do it. And, and I, that can well happen. I, I am not predicting the Yudkowsky scenario. Uh, it's just that I can't, I can't get the probability of it down to a level so low that I don't think it's worth worrying about.

    11. CWChris Williamson1:19:51

        I'm gonna take that as a white pill, even though you didn't know what that meant at the start of this conversation.

    12. RWRobert Wright1:19:55

        That was a white pill. That-

    13. CWChris Williamson1:19:55

        Yeah, that's hopeful

    14. RWRobert Wright1:19:56

        ... that was my first, my first white pill. Let me-

    15. CWChris Williamson1:19:57

        Your first ever white pill. I popped your white pill cherry.

    16. RWRobert Wright1:19:59

        Thank you for that, Chris. No, it felt so good.

17. 1:20:02 – 1:21:04

    Where to Find Robert

    1. CWChris Williamson1:20:02

       You're welcome. Robert Wright, ladies and gentlemen. Dude, you rule. I love all of your work. Everyone should go and read The Moral Animal. It's over 30 years old now and still just whole... It's so good. It's so fantastic. And you've got your new one as well. Where should people go to check out everything else that you've got going on?

    2. RWRobert Wright1:20:14

       Well, I have a newsletter called Nonzero, uh, on Substack, podcast called Nonzero on Twitter. I am @RobertWrighter. That's W-R-I-G-H-T-E-R. Kind of a pun. Um, and, uh, that's about the, some of it, the books, The God Test, uh, and, uh, and, uh, Let's, Let's Focus.

    3. CWChris Williamson1:20:37

       I am going to OpenAI's campus and HQ next week.

    4. RWRobert Wright1:20:41

       Really?

    5. CWChris Williamson1:20:41

       So I'll see if I can find out, I'll see if I can find out any, uh, any super secret insights there.

    6. RWRobert Wright1:20:45

       Do. R- please report back to all of us.

    7. CWChris Williamson1:20:47

       I shall indeed. Robert, appreciate you, man. Until the next time.

    8. RWRobert Wright1:20:50

       Thank you.

    9. CWChris Williamson1:20:51

       Catch you later on. Bye, everyone. Thank you very much for tuning in. If you enjoyed that episode, YouTube knows who you are deeply. It thinks you're gonna like this one even more. Go on, press it.

Episode duration: 1:21:05