Mustafa Suleyman: Will AI Save Humanity or End It?

1. 0:00 – 1:42

   Introduction

   1. SPSpeaker0:00

      So I feel like people fall in one of two camps on AI. They either think it's gonna destroy all of humanity. 42% of CEOs surveyed fear artificial intelligence could destroy humanity.

   2. TNTrevor Noah0:12

      This is something that, put in the wrong hands, could destroy humanity.

   3. SPSpeaker0:15

      Or they think it's gonna solve every single problem. Mustafa Suleyman. Mustafa Suleyman.

   4. TNTrevor Noah0:20

      Mustafa Suleyman is an artificial intelligence pioneer. He is the AI CEO at Microsoft. He is very big in the artificial intelligence world.

   5. SPSpeaker0:29

      How do we manage these technologies so that we can coexist with them safely?

   6. TNTrevor Noah0:35

      Can humans and AI coexist with each other peacefully without one taking over the other? [upbeat music] This is What Now? With Trevor Noah. [upbeat music] Mustafa, how are you, man? I'm very good, man. This is great. This is good. It's funny that there's, there's, there's almost two or three different types of conversations I have with people. There's ones where I'm hanging out, it's my friends, we're discussing just whatever, you know, shooting the shit. Then there's some where I bring a person in and I'm, I'm trying to, like, get something from them or learn about their world. And then there's the third type of interview that often stresses me the most because I feel like I'm speaking to a person who is, who has, like, an outsized influence on our world. [laughs] And if I mess it up, I don't ask the questions that the world needs. And I feel like you're one of those people because even before your current job, you were considered

2. 1:42 – 3:36

   Early DeepMind

   1. TNTrevor Noah1:42

      one of ... Like, if there was a, a Mount Rushmore of- [laughs] ... the founders of AI, with none of the, with none of the baggage, with none of the baggage of Mount Rushmore. No colonial history. No colonial history. You can all be good on that one. [laughs] Yeah, no colonial history. But if there was, like, a large Mount Rushmore, your face would be up there, you know, as being part of DeepMind and the founders of DeepMind. And then now you are helping Microsoft, like, one of the biggest, you know, tech companies in the world by, you know, market cap and just by influence, shape its view on AI. And so maybe, maybe that's where I wanted to start this conversation because it's, it almost feels like where we meet you in the journey now, you know? What would you say has been the biggest shift in, in your life and what you've been doing in AI, going from a startup that was on the cusp in this fledgling world of AI to now being at the head of what's gonna shape all of our lives in AI? Wow, what an opener. I mean, seriously, no pressure, huh? [laughs] You know, the crazy thing is that I've just got incredibly lucky. I mean, I was strange enough to start working on something that everybody thought was impossible, was totally sci-fi, that was, you know, just dismissed by even the kind of best academics, let alone any of the big tech companies didn't take it seriously. I mean, 2010, you know, just to really ground on that, we had just got mobile phones three years earlier. Yeah. The App Store was just coming alive. You couldn't even easily upload a photo from your phone to, you know, an Apple, the cloud, right? And somehow, you know, my courageous, visionary co-founders, Demis Hassabis and

3. 3:36 – 6:36

   How AI Works

   1. TNTrevor Noah3:36

      Shane Legg, had the foresight to know that, you know, technology ultimately, digital technologies ultimately become these learning algorithms which, when fed more data and given more compute, have a very good chance of learning the structure and nature of the universe. And so I was just very privileged to be friends with them, be part of that mission to ... You know, I was only 25 years old and, um, you know, kind of had the fearlessness to believe that if we could create something that truly understood us as humans, then that actually represents one of the best chances we have of improving the human condition. Hmm. And that's always been my motivation, to create technologies that actually serve us and make the world a better place. Like, I was cheesy before it was cheesy. [laughs] Pre-cheese. [laughs] You said something that, like, that sparked a thought in my mind, and I think a lot of people would love to better understand this. We see headlines all the time saying, "AI this, AI in your job, AI does the, the, the thinking, the not thinking, the ..." And you said something that engineers will gloss over quite quickly. You'll go, "Data and compute, and then a model, and then a ..." Help, help me break that down. Help me ex- ... Like, just explain that to me in the simplest terms possible. What changed, and, and what are you actually doing? Because it's, it's like we always had data, right? We, we, we've had documents, we've had files, we've had information. We always had computers. Well, not always, but we had computers for decades. What, what changed, and what is AI actually coming from? So the, the best intuition that I have for it is that our physical world can be converted into an information world. And information is basically this abstract idea, like it's mathematics, it doesn't exist in the physical world, but it's a representation of the physical objects. And the algorithm sounds complicated, but really it's just a mechanism for learning the structure of information and the relationship of one pixel to another pixel, or one word to another word, or one bit in the audio stream to the next bit in the audio stream. So I know that sounds very, like, abstract, but the structure of reality or the structure of information is actually a highly learnable-

   2. SPSpeaker6:14

      Sort of function, right?

   3. TNTrevor Noah6:16

      Mm-hmm.

   4. SPSpeaker6:16

      And that's what we saw in the ve- very early part of AI between 2010 and sort of 2016. These models could learn to understand, or at least not understand, but maybe they could learn to generate a new image just by reading a million images.

4. 6:36 – 9:00

   Do Machines Think?

   1. SPSpeaker6:36

      And that meant that it was learning that, you know, if an eye was here and another eye was there, then most likely there would be some form of a nose here. And although it didn't apply the word nose and eye, just had a statistical correlation between those three objects, such that if you then wanted to imagine, well, where would the mouth be, it wouldn't put the mouth in the forehead. It would put the mouth just below the nose. And that's what I mean about the structure of information. The algorithm learns the relationship between the common objects in the training data, and it did that well enough that it could generate new examples of the training data. First, in 2011, it was handwritten black and white digits. Then by 2013 it was, like, cats in YouTube images, which was what Google did back in 2013. Then as it got better, it could do it with audio, and then over time, you know, roll forward another 10 years, it did it with text. And so it's just the same core mechanism for learning the structure of information that has scaled all the way through.

   2. TNTrevor Noah7:36

      So it's interesting, I, I, I heard you say what it does, and I also noticed at a moment you said it understands, and then you said-

   3. SPSpeaker7:44

      Yeah

   4. TNTrevor Noah7:44

      ... well, no, wait. And I, I've actually noticed quite a few engineers and people who work in AI and tech struggle with explaining it to layman's using, you know, human language. But then very quickly going like, "No, no, no, no, no, it's, it's, it's, uh, not human language." It's like, it- does it think or does it do what we think thinking is?

   5. SPSpeaker8:05

      Yeah, I mean, this is a profound question, and basically it shows us the limitations of our own vocabulary. Because what is thinking? You know, i- that it sounds like a silly question, but it's actually a very profound question. What is understanding? If I can simulate understanding so perfectly that I can't distinguish between what was generated by the simulation and what was, what was generated by the thinking or understanding human being, then if those two outputs are equivalently impressive, does it matter what's actually happening under the hood?

   6. TNTrevor Noah8:45

      Yeah.

   7. SPSpeaker8:45

      Whether it's thinking or understanding or whether it's conscious, you know, it's a very, very difficult thing to ask because we're kind of behaviorists in the sense that as humans, we trust each other, we learn from each other,

5. 9:00 – 11:04

   Humanist AI

   1. SPSpeaker9:00

      and we connect to each other socially by observing our actions. You know, I, I don't know what's happening inside your brain, behind your eyes, inside your heart, in your soul, but I certainly hear the voice that you give me, the words that you say, I watch the actions that you do, and I observe those behaviors. And the, the really difficult thing about the moment that we're entering with this new AI agentic era, as they become not just pattern recognition systems, but whole agents, is that we have to engage with their behaviors increasingly as though they're like sort of digital people.

   2. TNTrevor Noah9:36

      Hmm.

   3. SPSpeaker9:36

      And this is a threshold transformation in the history of our species. 'Cause they're not tools, they're clearly not humans, they're not part of nature. They're kind of a fourth relation, a fourth emergent kind of... I don't know how to describe it other than a fourth relation.

   4. TNTrevor Noah9:54

      Yeah, I mean, you- you've called AI the most-

   5. SPSpeaker9:57

      Mm

   6. TNTrevor Noah9:57

      ... powerful general purpose technology that we've ever invented, and when I, when I read that line in your book, I was thinking to myself, I was like, man, you are now at the epicenter of helping Microsoft shape this at scale, you know? Uh, and, and then it made me wonder, what are you actually then trying to build? Because everyone has a different answer to this question, I've realized.

   7. SPSpeaker10:22

      Oh.

   8. TNTrevor Noah10:22

      You know, if you, if you ask Sam Altman, ChatGPT, Sam Altman says, "I'm trying to build artificial general intelligence." And I go like, "Oh, I like the app." He's like, "I don't care about the app, actually."

   9. SPSpeaker10:33

      Yeah.

   10. TNTrevor Noah10:33

       "I wanna make the God computer." And then you speak to somebody else and they say, "Oh, I'm trying to make AI that can help companies. I'm trying to make AI that helps with..." So what are you actually trying to build?

   11. SPSpeaker10:45

       I care about creating technologies that reduce human suffering. I want to create things that are truly aligned to human interests. I care about humanist super intelligence.

   12. TNTrevor Noah11:01

       Hmm.

   13. SPSpeaker11:01

       And that means that at every single step,

6. 11:04 – 14:53

   Philosophy vs. Tech

   1. SPSpeaker11:04

      new inventions have to pass the following test. In aggregate net-net, does it actually improve human wellbeing, reduce human suffering, and overall make the world a better place?

   2. TNTrevor Noah11:17

      Yeah.

   3. SPSpeaker11:17

      And it seems, like, ridiculous that we would want to apply that test. Like, surely we would all take it for granted that no one would want to invent something that causes net harm.

   4. TNTrevor Noah11:27

      Yeah.

   5. SPSpeaker11:27

      Right? But you know, there's certainly been other inventions in the past that we could think of that, you know, arguably have delivered net harm, right? And we have a choice about what we bring into the world, and so even though it's in the context of Microsoft, the most valuable company in the world today, we have to start with values and what we care about. And to me, a humanist super intelligence is one that always puts the human first and works in service of the human. And obviously there'll be a lot of debate and interpretation over the next few decades about what that means in practice, but I think it's the correct starting point.

   6. TNTrevor Noah12:02

      I've al- I've always wondered how the two sides of your brain, um-

   7. SPSpeaker12:06

      [laughs]

   8. TNTrevor Noah12:08

      ... sort of wrestle with each other around these topics because, uh, you know, someone asked me, they were like, "Oh, who are you having on?" I go, "Mustafa's coming on. Mustafa Suleyman." And they're like, "Oh, what does he do?" And I explained a little bit and I was like, "Oh, so like a, like an AI guy." Then I was like, "Yeah, but he's also a philosopher." And they're like, "What do you mean as a philosopher?" And then I was like, "No, no, no, like actually. Like actually this is somebody who has studied philosophies, engaged it." Like you, you, you, you think about the human ramifications of the non-human technologies that are being built by and for humans. And, and what, you, you know what it is for me is I, I, I always judge people by what they choose to yada, yada, if that makes sense. You know? So I've talked to some people in tech, and I say, "What about, what about the dangers?" And they go, "Oh, look, I mean, of course we, we've got to be aware of the dangers, but, but the future and it's so big."

   9. SPSpeaker12:58

      [laughs]

   10. TNTrevor Noah12:58

       And then I remember once I, I, I met you. The first time actually I met you, I said, "Uh, the technology's amazing." And then you went, "The dangers. Let me tell you about the dangers. Let me tell you about the things we need to consider. Let me..." And I was like, "What just happened here?"

   11. SPSpeaker13:12

       [laughs]

   12. TNTrevor Noah13:13

       Do you know what I mean? I was just like, "What, what, is, is this guy working against himself?" And, and so I wonder now, like when you're in that space, when you're working on something that is that big, how do you find the balance? Because we would be lying if we said humans could live in a world where we could ignore technologies. I've seen people say that. My opinion is that you can't ignore a technology, right? You can't just be like, "No, we'll act like it doesn't exist." But on the other hand, we also can't act like the technology's inevitable, because then we've given ourselves up. So when you're the person who's actually at the epicenter of trying to build our future, and I know it's not you alone, please don't get me wrong, but how do you, how do you think about that? How do you grapple with philosophy versus business, philosophy versus technology, human versus like an outcome? What, what are you thinking of?

   13. SPSpeaker14:01

       You, you've called out my split-brain personality, and now I'm like pinging-

   14. TNTrevor Noah14:05

       [laughs]

   15. SPSpeaker14:05

       Which, which side of me should answer? [laughs]

   16. TNTrevor Noah14:07

       You can answer twice.

   17. SPSpeaker14:08

       Oh, yeah. [laughs]

   18. TNTrevor Noah14:09

       You can answer twice.

   19. SPSpeaker14:09

       You can pick your answer. I'll give both. [laughs] No, I think part of it is just being English, you know?

   20. TNTrevor Noah14:16

       Oh.

   21. SPSpeaker14:16

       I'm kind of like, I'm, I'm more comfortable than the average American thinking about the kind of cynical dark side of things.

   22. TNTrevor Noah14:25

       It's those rainy days in there.

   23. SPSpeaker14:26

       It's rainy days, man.

   24. TNTrevor Noah14:27

       It's those rainy days.

   25. SPSpeaker14:29

       [laughs] And I just think, I don't know, like truth exists in the honesty of looking at all sides. And I think if you have a kind of bias one way or another, it just doesn't feel real to me.

   26. TNTrevor Noah14:44

       Mm.

   27. SPSpeaker14:44

       And I guess that's kind of my philosopher or kind of academic side that is a core part of who I am.

   28. TNTrevor Noah14:50

       Yeah.

   29. SPSpeaker14:51

       Like I, I'm comfortable,

7. 14:53 – 17:02

   Future of Energy

   1. SPSpeaker14:53

      you know, living in, you know, what to some people might seem like a set of contradictions because to me, they're not contradictions, they're truth manifested in a single individual. But if you are honest about it, it's also manifested in every single one of us, too. Um, you know, I happen to be in a position, but like the company has to wrestle with these things. Our governments have to wrestle with these things. Every single one of us as citizens has to confront this reality because, you know, every single technology just accelerates massive transformation, which can deliver unbelievable benefits and also create side effects. And it's like that, that idea has been repeated so many times, it now kind of sounds trite. But once you get over the trite part, you still have to engage with the fact that the very same thing that is going to reduce the cost of production of energy over the next two decades by 100X, reduce the cost of energy by 100X-

   2. TNTrevor Noah15:49

      You think that's what AI can do?

   3. SPSpeaker15:51

      In two decades. 100%. Like, I feel very optimistic about that.

   4. TNTrevor Noah15:53

      Wait, wait. So now say that again.

   5. SPSpeaker15:54

      So reduce the cost of energy. I think energy is going to become a pretty much cheap and abundant resource. I mean, even solar panels alone are probably gonna come down by another 5X in the next 10 years. Like, just that breakthrough alone is gonna reduce the price of most things. Um, and, and-

   6. TNTrevor Noah16:13

      And what, what is that, what is that through? Is that like the AI being more efficient, teaching us how to create different energy grids, teaching us how to create energy differently? Like what, what, what would you predict it coming from?

   7. SPSpeaker16:23

      Well, I mean, so at, at the most abstract level, these are pattern matching systems that find more efficient ways than we are able to invent ourselves as humans for combining new materials. Now, that might be in grid management and distribution. It might be inventing new synthetic materials for, um, you know, batteries and storing-

   8. TNTrevor Noah16:43

      Oh, yeah

   9. SPSpeaker16:43

      ... renewable energy.

   10. TNTrevor Noah16:44

       Yeah.

   11. SPSpeaker16:44

       It might be in more efficient solar volta- um, you know, voltaic cells that can actually capture more per square inch, for example. I mean, there are so many breakthroughs that, you know, we're kind of on the cusp of that require just one or two more pushes to get them over the line, even the superconductors from last year.

   12. TNTrevor Noah17:01

       Mm.

8. 17:02 – 20:15

   AI’s Environmental Cost

   1. SPSpeaker17:02

      Those things, any one of those could come in, right? And if they do, we see massive transformation in the economy. I mean, imagine if by 2045, you know, energy is, let's say, 10 to 100X cheaper. We will be able to desalinate water from the ocean-

   2. TNTrevor Noah17:20

      Hmm

   3. SPSpeaker17:20

      ... um, anywhere, which means that we would have clean water in places that might be 50 degrees or whatever, you know, 120 degrees hot, right? Which means that we can grow crops in arid environments, which will mitigate the flow of migration because of climate change, which means that we could run AC units in places that we never could before. You know, there are so many knock-on effects of fundamental technologies, general purpose technologies like energy coming down by 10 to 100X. Um, so there are huge reasons to be optimistic that everybody is gonna get access to these technologies and the benefits of these technologies over the next couple decades.

   4. TNTrevor Noah17:57

      Yeah.

   5. SPSpeaker17:58

      And that will make life much easier and much cheaper for everybody on the planet.

   6. TNTrevor Noah18:03

      So let, let's, um, let's jump into that a little bit, like it could make energy how many times cheaper?

   7. SPSpeaker18:10

      Well, I was saying 100X cheaper-

   8. TNTrevor Noah18:12

      Yeah

   9. SPSpeaker18:12

      ... over 20 years.

   10. TNTrevor Noah18:13

       100X cheaper over 20 years. So this, this is, this is one of those instances that I've struggled with because, you know, like depending on where you get information and how you get information, it changes how you perceive the issue, right? So I remember being really angry when I saw how much water is consumed by like typing one query into ... co-pilots, ChatGPT, any, any AI model. Then I was even more angry when I saw how much water is consumed by, like, getting a picture, you know, made. And then I saw something else that was like, oh, this is nothing compared to cars and, you know, uh, produce and, like, making hamburgers and that. And then I was like, okay, like, where's the information coming from? Where's it not coming from? The response to the price of, of, of AI, like, is it, is it driven by the AI industry saying, "No, this is actually not that bad," or, like, how do you think we should look at it? Or how do you look at it?

   11. SPSpeaker19:10

       I mean, look, it, it consumes vast amounts of resources, precious metals, electricity, water. No question about that, right? Um, on the energy side of things, all of the big companies now are almost entirely 100% renewable. Certainly Microsoft, 100% renewable. I think we have 33 gigawatts of renewable energy in our entire-

   12. TNTrevor Noah19:32

       Oh, damn

   13. SPSpeaker19:33

       ... fleet of computation. Uh, for comparison, I think Seattle consumes two gigawatts per year of energy, so just to put that into perspective.

   14. TNTrevor Noah19:42

       The whole of Seattle-

   15. SPSpeaker19:43

       The whole of Seattle

   16. TNTrevor Noah19:43

       ... consumes two gigawatts?

   17. SPSpeaker19:45

       Yeah. [laughs]

   18. TNTrevor Noah19:45

       And Microsoft is creating how much?

   19. SPSpeaker19:47

       33 overall in the fleet.

   20. TNTrevor Noah19:49

       Wow.

   21. SPSpeaker19:49

       This is worldwide.

   22. TNTrevor Noah19:49

       Yeah, no, no, no, but still.

   23. SPSpeaker19:51

       And the vast majority of it is 100% renewable, so coming from solar or wind or, or, or water. But it also consumes a lot of water in the process. Like, we have to cool these systems down and, you know, for sure that consumes a lot. Now, I don't know that there is a easy way of, you know... There's, there's no shortcut there. It's expensive. It consumes,

9. 20:15 – 21:42

   Scale & Risk

   1. SPSpeaker20:15

      you know, resources. It consumes a lot of resources from the environment. Um, but I think net-net, when you look at the beneficial impact, to me it's justified. Like, you know, we, we... You wouldn't give up your car or tell people to give up their car anytime soon because it uses aluminum and rubber and... This is an essential part of your existence, and I think AI is gonna become an essential part of everybody's exi- existence and justify the environmental costs even though that doesn't mean that we have to go and consume diesel generators and, you know, carbon emitting. Like, we, we get to start again from scratch-

   2. TNTrevor Noah20:49

      Mm-hmm, mm

   3. SPSpeaker20:49

      ... which is to say new technology arrives, new standard has to be applied to it, which means that our water has to also be cleaned and recycled, which, you know, many of the data centers do now, you know, take full life cycle responsibility for cleaning the water. And the same with the energy. It has to be renewable. So there's, there's no easy way out. It's just a rough reality that producing things at this scale is definitely gonna consume more resources from the environment.

   4. TNTrevor Noah21:15

      It's funny, every time I try and think of it, I, I think of the, um, the gift and curse that comes with anything that scales. You know, I, I... The analogy I will always use for myself is I'll go, I think of, like, an airplane. Before an airplane's invented, especially, like, a, a large jumbo jet, the amount of people who can die while being transported somewhere is, is much lower really, if we're honest. You know, a car, four people, six people, whatever it might be, still tragic, but

10. 21:42 – 29:28

    Jobs vs. Workers

    1. TNTrevor Noah21:42

       a smaller number. The plane comes along, you can go further, you can go faster, but it also means there can be something more devastating on the other side of that plane crashing or something going wrong. And it feels like that scales with, with AI as well. It sounds like you're saying to me on the one side of it, this technology could completely change our relationship with economies and finance and society, but then there's, there's the looming other side of it that, that could crash. And so maybe that's a good place for us to start, like, diving into this is what's noise and what's very real for you as somebody who sees it? Because everyone gets a different headline about AI. Doesn't matter where you are in the world. It doesn't matter your religion, your race, whatever it is, everyone gets a different headline about AI. But when you're looking at it as somebody who is working on creating it every single day, what is real and what is noise in and around AI?

    2. SPSpeaker22:40

       So I think it's pretty clear to me that we're gonna face mass job displacement sometime in the next 20 years because whilst these technologies are for the first part of their introduction augmenting, like, they add to you as a human-

    3. TNTrevor Noah22:58

       Yeah, it's like a bionic leg

    4. SPSpeaker22:59

       ... they save you time. Yeah, it's like a bionic leg, but for, like, cognitive laborers.

    5. TNTrevor Noah23:04

       Hmm.

    6. SPSpeaker23:05

       You know? [laughs] Uh, I think, like, you could... You know, who was it? I think it was Steve Jobs that called it, like, the bicycle for the mind, you know? It's this sort of exercising. You know, digital technologies allow you to exercise new parts of your mind that you didn't know-

    7. TNTrevor Noah23:18

       Yeah

    8. SPSpeaker23:19

       ... you had access to, and I think that's definitely true, but much of the work that people do today is quite routine and quite predictable. It's kind of mechanized, um, yeah, like cognitive manual labor. And so that stuff, the machines are gonna get very, very good at those things, and the benefits to introducing those technologies are gonna be very clear for the company, for the shareholder, for the government, for, you know. And so we'll see, like, rapid displacement and people have to figure out, okay, what, what is my contribution to the labor market? I think those fears are very real, and that's where governments have to take a strong hand because there needs to be a mechanism for taxation redistribution. Taxation is a tool for incentivizing certain types of technologies to be introduced in certain industries. And so it's not just about generating revenue-

    9. TNTrevor Noah24:12

       Yeah

    10. SPSpeaker24:12

        ... it's about limiting, adding friction to the introduction of certain technologies so that we can figure out how to create other opportunities for people as this transition takes place.

    11. TNTrevor Noah24:22

        Yeah, it's funny, one of, one of my favorite quotes I ever, uh, heard was, um, I think it was Sweden's head of infectious diseases. I, I think that's what his job was. I spoke to him during the pandemic- And we were just talking about life in Sweden and what they do. And, and I asked him a question about labor and Sweden and how everything works out there, and, and he said something fascinating. It was, um ... No, I think he actually was in the labor department on that side. He said, "In Sweden," unlike in America, he said, "In Sweden, we don't care about jobs. We care about the workers." And I remember that breaking my mind because I went, oh yeah, everyone always talks about, like, the job as if the job is something that is affixed to a human being. But really, the human is the important part of the equation.

    12. SPSpeaker25:09

        Yeah.

    13. TNTrevor Noah25:09

        The job is just what the human does, and so our focus has to be on making sure that the human always has a job. But f- from what you're saying, we don't know what the job will be because the jobs that we know now are sort of easy to replace.

    14. SPSpeaker25:23

        Yeah.

    15. TNTrevor Noah25:24

        It's data entry, data capturing, sending an email, doing an Excel spreadsheet. That stuff is easy, actually, when it comes to AI. And then now we don't know what the next part of it is. And so maybe my next question to you then is when you're in that world, the philosopher side of your brain, like, what do, what do you think the onus is on us and the ... Like, I mean the tech companies and all the ... to work on discovering what the new job is, or do we not know what it'll be?

    16. SPSpeaker25:49

        Well, but also, I would tweak what you said, that, uh, the job of society or the, the function of society is to create jobs that are meaningful for people.

    17. TNTrevor Noah25:58

        Yeah.

    18. SPSpeaker25:58

        Like, I'm not sure I buy that. Like, I think-

    19. TNTrevor Noah26:01

        Oh, really?

    20. SPSpeaker26:02

        ... many people do jobs which are super unfulfilling, and that they would be quite happy to give up if they had an income.

    21. TNTrevor Noah26:09

        [laughs] Yeah, but this is true.

    22. SPSpeaker26:10

        Like, and so-

    23. TNTrevor Noah26:11

        No, this is true. Yeah, this is true

    24. SPSpeaker26:11

        ... like we're probably very lucky that we get to, we get paid for the thing that we would be doing if we didn't get paid.

    25. TNTrevor Noah26:16

        No, this is true. Yeah.

    26. SPSpeaker26:16

        Right? I would certainly be doing that. And so I think the function of society is to create, um, a peaceful, supportive environment for people to find their passion and live a life that is fulfilling. That doesn't necessarily have to overlap with a job or work. I would ... I, I mean, maybe I'm too much of a utopian, but I dream of a world where people get to choose what work they do and have true freedom. And, and people get tense about that idea, 'cause they're like, you know, "Work is about identity, and this is my role in society, and this is what is meaningful to me. And if I didn't have my job, I would be ..." It's like, nah, come on, man. Take a minute to think seriously. If you didn't have to work today, what would you do with your life? This is one of my favorite questions that I, I always ask people. If you didn't have to worry about your income, what would you do? And, you know, if you get into the habit of asking that question, people say some crazy things.

    27. TNTrevor Noah27:14

        Yeah.

    28. SPSpeaker27:14

        It's so inspiring.

    29. TNTrevor Noah27:15

        Yeah.

    30. SPSpeaker27:16

        And so yeah, maybe I'm a utopian dreamer, but I do think that is a relevant question for us to think about by 2045. I think it's a real chance that if we get this technology right, it will produce enough value, you know, aggregate value to the world, both in terms of the reduction of the cost of stuff-
11. 29:28 – 32:07

    Identity & Work

    1. SPSpeaker29:28

       Mm.

    2. TNTrevor Noah29:28

       And then I remember back to times when I'd be in a township in South Africa, or even in what we call the homelands, where our grandmothers would live, and, you know, that was the, the extended family, people literally living in huts, and it's dirt roads. And, and everyone would go, "Oh, what a terrible way to..." But I'll tell you now, there was no homeless person there. There was no one, like, stressing about a job in the same way. I'm not saying nobody wanted a job, but the gap between them thinking they didn't exist because there was no job was a lot greater than the people who were living in a world where, you know, your job was you. And so it's interesting that you say that because I, I do wonder how easy it'll be for us to grapple with it, like, like what that time will be.

    3. SPSpeaker30:08

       But it also just shows how much variation there is. You know, we come from ... You know, in terms of how humans live their lives.

    4. TNTrevor Noah30:15

       Yeah.

    5. SPSpeaker30:15

       I feel like we come from, you know, whatever our different backgrounds, we're still quite Western-centric, and we're just sort of quite homogenous. That, you know, we've, like, sort of had 300 years of specialization-

    6. TNTrevor Noah30:28

       Mm. Mm. Mm

    7. SPSpeaker30:28

       ... education, Protestant work ethic, atomization of families, smaller and smaller units, spread ... You know, leave your home, you know, sort of physical locale-

    8. TNTrevor Noah30:38

       Yeah. Yeah

    9. SPSpeaker30:38

       ... where your community is. And- I think there's a kind of loneliness epidemic as a result. Like, I feel, you know, you, you probably, like me, you know, pour your life and soul into your work, and then what was ... I guess, what was it like for you when you switched your job, right? Like, 'cause that was obviously a massive part of your identity. It was what you did every day-

    10. TNTrevor Noah30:57

        Yeah

    11. SPSpeaker30:58

        ... 24/7.

    12. TNTrevor Noah30:59

        Yeah. Uh, uh, but you see, to that point, I left The Daily Show to go spend more time in South Africa at home.

    13. SPSpeaker31:07

        Mm.

    14. TNTrevor Noah31:08

        And, y- you know, one of my best friends had a beautiful, um, phrase that he said to me. He said, um, "In life, sometimes you have to let go of something old to hold onto something new."

    15. SPSpeaker31:18

        Mm.

    16. TNTrevor Noah31:19

        It's not always apparent what the value is of, like, of something that we're sacrificing. It's not always apparent. But if we are unable to assign that value ourselves, we'll get stuck. So leaving The Daily Show, I leave a ton of money behind. I leave, you know, the, the status, the everything, but no one has assigned a value to my friends.

    17. SPSpeaker31:39

        Mm.

    18. TNTrevor Noah31:39

        No one's assigned a value to my family. No one's assigned a value to the languages that people speak to me in my country. There's no Economist article on that, so I don't know what the value of that is. Someone can look at my bank accounts and go like, "That's value," but they don't tell me what my friends' actual value is. And so I think that's where, you know, it's, it's, it's hard. I, I just had to, like, decide it for myself, and I think we, we all have to, but I think some people won't have the luxury because of how,

12. 32:07 – 37:22

    Speed of AI

    1. TNTrevor Noah32:07

       you know, how close they are to the line.

    2. SPSpeaker32:10

       And we have s-

    3. TNTrevor Noah32:11

       You know, when you talk about those jobs that are gonna disappear, there's somebody who's going, "I don't have the luxury of pontificating-

    4. SPSpeaker32:16

       Exactly right

    5. TNTrevor Noah32:17

       ... because tomorrow is, is what's coming. I can't think about, like, ah, what will be," and that's, like, a real luxury, I, I think.

    6. SPSpeaker32:24

       And I think that's why talking about the dangers and the fears now is so important because this is happening super fast. I mean, the transition has surprised me and all my other peers in the industry in terms of how quickly it's working.

    7. TNTrevor Noah32:40

       Mm.

    8. SPSpeaker32:40

       And at the same time, you know, we, we've, we've, we're also kind of, like, unsure about [laughs] whether s- the nation state is gonna be able to sort of respond to the transition too because, you know, y- y- you're maybe lucky because you already had enough income that you didn't have to worry about it, and you could ... It was really just, like, connecting to your heart.

    9. TNTrevor Noah32:58

       Yeah. Yeah.

    10. SPSpeaker32:58

        But many people are gonna be like, "Well, I'm, I'm gonna have to still be able to provide food for my family and carry on doing my work throughout this crazy transition."

    11. TNTrevor Noah33:07

        So then let me ask you this. Y- you see, that, that, that, that is such an interesting thought. You and your peers were shocked and are shocked at the rate of how AI is going and growing. To me, that, that blows my mind 'cause I go like, "Of course I'm shocked."

    12. SPSpeaker33:22

        Yeah.

    13. TNTrevor Noah33:22

        I don't know how to code. You, you get what I'm saying? Of course I'm gonna be shocked. But now when you say that, I then wonder, as somebody who's been ... You are truly an OG in the game of AI. Like, really, really. You're not, like, one of the people who's jumped on now because it's blowing up. You were in it before there was money.

    14. SPSpeaker33:37

        Yeah.

    15. TNTrevor Noah33:37

        And now you're in it in the thick of things. Where do you think we are in, in AI's development? Are we, are we looking at a baby? Are we looking at a teenager? Are we looking at, like, a, you know, 20-something year old? Like, w- where do you, where do you think we are when we look at AI's development?

    16. SPSpeaker33:51

        I think part of the challenge with the human condition in this mad, globalized, digitized world is that we're so overwhelmed with information, and we're so ill-equipped biologically to deal with scale and exponentials. Like, it's just very f- like, when I say 2045, like, I'm just used to living in 2045.

    17. TNTrevor Noah34:13

        [laughs]

    18. SPSpeaker34:13

        Like, it's just my weird ... Like, I've always been like that.

    19. TNTrevor Noah34:16

        [laughs]

    20. SPSpeaker34:16

        And it's kind of become second nature for me to casually drop that. But, you know, I, if I do that with some random people I meet at the bar, I'm obviously just a freak. There's just no one thinks about that. People barely think about what they're gonna do in two weeks-

    21. TNTrevor Noah34:28

        Yeah. Yeah

    22. SPSpeaker34:28

        ... let alone 20 years. So, and, and likewise, you know, people are sort of not equipped to think of what does an exponential actually mean? Now, I'm lucky enough that I got a, a practical intuition for the definition of an exponential because between 2010 and 2020, for 10 years, me and a bunch of other random people worked on AI, and it was sort of working but basically didn't work, the flat part of the exponential. Even though we could see little doublings happening every 18 months, it started from a base of, like, alm- effectively zero.

    23. TNTrevor Noah35:05

        Yeah.

    24. SPSpeaker35:05

        Think of it. And so it isn't until the last few doublings that you see this massive shift in capability.

    25. TNTrevor Noah35:13

        Mm.

    26. SPSpeaker35:14

        I mean, for example, like, four years ago, before GPT-3, a language model could barely predict the next word in one sentence. Like, it's just kind of random. It was a little bit off. Often didn't make sense.

    27. TNTrevor Noah35:26

        This is 2013?

    28. SPSpeaker35:28

        No. This is 2020, three or four years ago.

    29. TNTrevor Noah35:30

        2023?

    30. SPSpeaker35:31

        So, no, no, not 2023. Three or four years ago.
13. 37:22 – 44:07

    Containment Challenge

    1. SPSpeaker37:22

       specialized hand-coded programming for each data domain.

    2. TNTrevor Noah37:26

       Damn. I mean, the... You know what it, you know what it reminds me of? Have you ever s- have you ever seen that thing where they talk about when they, when they're trying to ex- explain exponential to you, there's one example they give, um, which is folding paper.

    3. SPSpeaker37:41

       Yeah.

    4. TNTrevor Noah37:41

       You know, so they go, if you fold a piece of paper in half, and then you fold it in half, and then you fold it in half, and you fold it in half, and you, and I think you can't do it more than seven times or something like... But then they go if you could keep folding it in half-

    5. SPSpeaker37:52

       That's it

    6. TNTrevor Noah37:52

       ... the number to get to, like, space is really small actually.

    7. SPSpeaker37:54

       I think it's like the 64th gets to the moon or something crazy. [laughs]

    8. TNTrevor Noah37:56

       Yeah, but, but I remember, I remember seeing that and I was like, wait, wait, wait, wait, wait, what? They're like, yeah, if you, if you do it one w- and then you do one more. And I was like, wait, 64? I was like, no, what do you mean? Like 64,000? They're like, no, no, no, 64. And, and, and that's when you start to understand how we just generally don't understand exponential and, you know, you know, and these, like, compound gains. And so now that's where I wanted to ask you about the idea of containment.

    9. SPSpeaker38:22

       Mm-hmm. Mm-hmm.

    10. TNTrevor Noah38:24

        Your book, of every one I've read, I mean, every one who's written about AI, who's, like, in it, in it, in it, was the only book that I would say spent the majority of its time talking about the difficulties of grappling with AI. Yeah, you talked about the beauty of what we could do with medicine and technology and, and we, we should get into that to talk about some of the breakthroughs that you made at DeepMind. But, like, containment seems like the greatest challenge facing us, and we don't even realize it and we don't really talk about it. Talk me through what containment means to you and why you think we should all be focusing on it.

    11. SPSpeaker39:05

        So the trend that's happening is that power is being miniaturized and concentrated, and it's being made cheap and widely available to everybody. Why do I say power? Because making an accurate prediction about how text is gonna unfold or what code to produce or what, you know, frame to extend given a video, that is power. Like, predictions are power. Intelligence is an accurate prediction given some new environment. That's really fundamentally what we do as humans. We're prediction engines, and these things are prediction engines. So they're gonna be able to make phone calls, write emails, use APIs, write arbitrary code, make PDFs, use Excel, act like a project manager that can do stuff for you on your behalf. So you as a creator or as a business owner, you're gonna be able to hire a team of AIs-

    12. TNTrevor Noah39:59

        Mm-hmm

    13. SPSpeaker39:59

        ... specialized in marketing or HR or strategy or whatever it is, or coding, and that's gonna give you leverage in the world. I mean, you said about the, the kind of, you know, the strange, like, function of scale. What this is gonna do is scale up every single individual and every single business to be able to deliver way, way more, because the cost of production is gonna be-

    14. TNTrevor Noah40:22

        Yeah

    15. SPSpeaker40:22

        ... you know, basically zero marginal cost. Now, on the one hand, that's amazing because it means the time between you having a creative idea and being able to prototype it or experiment with it in some way or even build it up to the max sale is gonna go to, shrink to basically, you know, nothing. You just think something, vibe code it up in natural language, produce the app, build the, you know, website, try out the idea that you have. But the flip side of that is that anybody can now not just broadcast their ideas, like we had with the arrival of podcasts or the arrival of blogs on the web before that. It meant that anyone could talk to everyone.

    16. TNTrevor Noah41:02

        Yeah.

    17. SPSpeaker41:03

        Which was amazing at one level.

    18. TNTrevor Noah41:04

        Because no one controlled the infrastructure in a way.

    19. SPSpeaker41:06

        Exactly. And it's super cheap for anybody to go publish a website-

    20. TNTrevor Noah41:10

        Yeah

    21. SPSpeaker41:10

        ... or do a blog or do a, do a podcast. So the same trend is gonna happen for the ability for people to produce stuff, do actions. So in, in social media, it was like anyone can now broadcast.

    22. TNTrevor Noah41:23

        Yeah.

    23. SPSpeaker41:23

        Now with AI, anyone can now take action. You can, like, build a business, you know, start a channel, create content, you know, whatever it is that you, you believe in. I mean, you, you might be a religious person and you're trying to evangelize for your, you know, or you're trying to persuade somebody of your political ideas. Everyone is gonna have a much easier time of executing on their vision. And obviously, the benefits of that are pretty clear, but the, the downside of that is that that inevitably causes conflict because we just disagree with each other.

    24. TNTrevor Noah41:57

        Yeah.

    25. SPSpeaker41:57

        We, you know, we don't hate each other. You're not evil, I'm not evil, but we've got different views. And if I can just kind of at the click of a button execute my crazy ideas, and you could execute your crazy ideas that are, like, practical actions affecting the real world, and everyone's doing the same thing, then inevitably that is gonna cause an immense amount of conflict. At the same time, the nation state, which is supposed to have a monopoly over power in order to create peace, that's the contract that we make-

    26. TNTrevor Noah42:24

        Ooh

    27. SPSpeaker42:24

        ... with the nation state.

    28. TNTrevor Noah42:24

        Oh, yeah. Damn.

    29. SPSpeaker42:25

        Nation state's getting weaker and kind of struggling, right? And so containment is a belief that completely unregulated power that proliferates at zero marginal cost is a fundamental risk to peace and stability. And it's an assumption that you have to gently restrict in the right way, um, mass proliferation of super powerful systems because of the one-to-many impact that they're gonna have.

    30. TNTrevor Noah42:53

        If I hear what you're saying correctly, it's almost like- It's almost like you're saying if something is hard to do, only a few can do it.
14. 44:07 – 47:25

    AI as Agent

    1. TNTrevor Noah44:07

       got into trouble or didn't. It, but there was like a, it's like we know where to go. Then you get cable news. It expands, becomes a lot harder now. Wait, who's saying the news? Who's saying the truth? Who's not saying the ... Do you punish them? But still you could go to them, you know? So somebody like Fox News can get sued for saying something about a Dominion voting system, but Dominion knew where to go. They went, "We're going f- after Fox News." So in a strange way, even in that world, the system is still sort of working because there's, there's friction, right? There's, it, it is where it is, and it has to be broadcast. Then the internet, streaming, YouTube, et cetera, you don't even know who the person is, where the person is, if it's a person. And then if they say something that's not true and it enrages the masses, where do we go?

    2. SPSpeaker44:52

       And it's not just that it's gonna say something, it's gonna do something.

    3. TNTrevor Noah44:55

       Ah, damn, Mustafa. Why you gotta ... Oh, man.

    4. SPSpeaker44:56

       It's gonna build the app. It's [laughs] gonna build the website. It's gonna do the thing. And so look, I think the, this is the point about confronting the reality of what's coming, right?

    5. TNTrevor Noah45:05

       Yeah, but wait, wait. Go, go back on that. You see, that's something I always forget. Oh, man. See, we, we always think of AI as just, like, saying. Talk, let's talk a little bit more about the doing 'cause that is what makes it unique, you know? And on one of the episodes, we had, um, uh, Yuval Noah Harari, the book-

    6. SPSpeaker45:22

       Yeah

    7. TNTrevor Noah45:22

       ... you know?

    8. SPSpeaker45:22

       Of course.

    9. TNTrevor Noah45:23

       Yeah.

    10. SPSpeaker45:23

        I'm good friends with Yuval. He's awesome.

    11. TNTrevor Noah45:25

        And Yuval, you know, was, was on for his book Nexus, and we're talking about information and systems.

    12. SPSpeaker45:30

        And stories.

    13. TNTrevor Noah45:31

        And stories.

    14. SPSpeaker45:32

        Yeah.

    15. TNTrevor Noah45:32

        And one of the things he kept going on about was he said, "I know AI's a tool, but we've never had a tool that makes itself." And you talk about that as well.

    16. SPSpeaker45:41

        Yeah.

    17. TNTrevor Noah45:42

        We've never had a hammer that makes the hammer without you getting involved. It's just make the thing, make the thing, make the thing. Atom bomb is one thing, but no atom bomb makes an atom bomb, you know? And so that doing-

    18. SPSpeaker45:51

        Well, there's a lot of ideas there. So there's, it, so firstly, actions are stories. And Yuval's point was that the history of civilization has been about creating stories.

    19. TNTrevor Noah46:06

        Yeah.

    20. SPSpeaker46:06

        Religious stories, historical stories, ideological stories, like stories of oppression, of domination, of persuasion. And it was really humans that had the ... And but it was, it was the friction of being able to pass on that story through spoken word and then through digitization which slowed down the spread of change. Um, and that was an important regulator and filter. So as we've talked about, the digitization speeds up the distribution of those stories, which allows that information to spread. But it's not just that. It also is an actual agent that is going to operate like a p- project manager in your little mini persuasion army. And people are gonna use those things not just for phishing attacks or for, you know, sort of selling stories, but for actually making the PowerPoint presentation, of building the app, of planning-

    21. TNTrevor Noah46:56

        Yeah

    22. SPSpeaker46:56

        ... you know, making the project plan. And so it's kind of o- operating just as a, you know, member of your team would. And I think that's where all the benefit comes from, but it's also where there's, like, massive risks at the same time. And then the other point that you made about, like, it can edit itself. This is a new threshold. It, you know, a technology that is able to observe what it produced, modify that observation. Like, it can critique its own image

15. 47:25 – 50:00

    Self-Improvement Risk

    1. SPSpeaker47:25

       that it produced and say, [clears throat] "Well, it looks like this part of the hand was kind of weak. Okay, so we'll generate another one." Or it'll produce a poem-

    2. TNTrevor Noah47:32

       Yeah. Mm

    3. SPSpeaker47:32

       ... or a strategy and then edit that and update it. And that's just editing its output. But it can also edit its kind of input, its own code, its own system processing in order to improve [laughs] with respect to its objective, and that's called recursive self-improvement, um, you know, where it can just iteratively, iteratively improve its own code with respect to some objective. And I've long said that that is a threshold which presents significantly more risk than any other aspect of AI development that we've seen so far. I mean, that really is a kind of subset of technologies that if we're really gonna focus on humanist super intelligence-

    4. TNTrevor Noah48:13

       Yeah

    5. SPSpeaker48:14

       ... being skeptical and critical and auditing the use and development of recursive self-improving methods, that's where I think there's genuine risk.

    6. TNTrevor Noah48:24

       We're gonna continue this conversation right after this short break. [upbeat music] So do you ever feel like you could be sitting in a position where you're sort of like the Oppenheimer of today? Do you ever feel like you're sitting in a position where you're both grappling with the need for the technology, but then also the not 0% chance that the thing could burn the atmosphere? Like, how do you, how do you grapple with that? I, I often wonder this. Even when I think of, like, engineers and, and people who are writing the code, I'm always fascinated by people who- Write the code for the thing that's gonna write the code. Like other people have jobs, but like if you told me, as Trevor, "Hey, Trevor, can you help this AI learn to do comedy?" I'd be like, "No."

    7. SPSpeaker49:15

       [laughs]

    8. TNTrevor Noah49:16

       Do you know what I mean? So I'm always intrigued by the coders who are making the thing that's now coding. I, like I, I just wanna know like how you, how you like wrestle with this entire thing. Do, do you think it's larger than us and we have to wrestle with it? Or w- like w- what is, what is that, what is that battle like for you?

    9. SPSpeaker49:32

       You know, in, in, in COVID, when I started writing The Coming Wave, my book, I was really motivated by that question. It was, it was, that was actually one of the core questions is, how does technology arrive in our world and why? What are the incentives, the system level incentives that default to proliferation, that just produce more and more? And it's very clear that there's demand to live better. You know, people want to have the cheaper T-shirt

16. 50:00 – 1:02:22

    Early Evangelism

    1. SPSpeaker50:00

       from, you know, the more affordable thing. You wanna have the cheaper car. You wanna be able to go on vacation to all the places around the world.

    2. TNTrevor Noah50:07

       Yeah.

    3. SPSpeaker50:07

       And so planes get cheaper and more efficient because there's loads of demand for them. And so it's really demand that improves the efficiency and quality of technologies to reduce their price so that they can be sold more. And that, that is why everything ultimately pro- proliferates and why it inevitably happens. Because we haven't really had much of a track record of saying no to a technology in the past. There's regulations around technology and restrictions, which I think have been incredibly effective if you think about it. You know, every technology that we have is, is regulated immensely. Flight or cars or emissions. I mean, you know, so, so people, I think particularly in the US, like have this sort of allergic reaction to the word regulation. But actually, regulation is just the sculpture of technologies, right? Chipping away at the edges and the, and the, the, the pain points of technology in the collective interest, and that's what we need the state for. The state has the responsibility for the common good, and that's h- why we have to invest in the state and build the state because it isn't in the interest of any one of the individual corporate actors or academic, you know, researchers, AI researchers-

    4. TNTrevor Noah51:18

       Yeah

    5. SPSpeaker51:18

       ... or anyone individually to really take responsibility for the whole.

    6. TNTrevor Noah51:24

       Hmm.

    7. SPSpeaker51:24

       And that's why we need governments more than ever, right? Not to hold us back or to slow us down, but to sculpt technology so that it delivers all of the benefits that we hope it will whilst limiting the potential harms that it can produce.

    8. TNTrevor Noah51:38

       I wanna take a step back and talk about your journey with AI. Today, in 2025, it seems obvious. Now when I speak to people, oh, everyone's like, "Oh yeah, AI, AI, AI." I wasn't even in the game when... I mean, I, and I was like an early just, you know, layman. I remember showing people at my office the first iterations of GPT-3 and DALL-E. I remember when DALL-E was still like, I mean, like basically tell it about an image and then just go away. Go, go book a vacation for a week, come back for your image and it would make... But even then I was like, "This is gonna change everything." People were like, "Oh, no, I don't know." And I remember struggling to convince people that this thing was going to be as big as it was going to be. I was doing this i- in like this timescale. When I look at your history, you, you literally have years of your life where you were in boardrooms-

    9. SPSpeaker52:38

       [laughs]

    10. TNTrevor Noah52:39

        ... telling world leaders, telling investors, telling tech people in Silicon Valley, "Hey, this is what the future of AI is going to be," and no one listened to you. Not no one zero, but I mean like no one listened-

    11. SPSpeaker52:54

        Pretty much

    12. TNTrevor Noah52:54

        ... to you, right? Now, that made me wonder two things. One, should we be worried about our world and our future being built by people who are unable to see the future? And two, what did you see then that we might not be seeing now?

    13. SPSpeaker53:12

        Shit, that's a hard question, man. I think wh- as you were talking, one of the i- um, memories that came to mind was, I remember in 2011, we had an office in Russell Square in the center of London, near U- University College London, UCL. And, uh, someone in the office showed me this handwriting recognition algorithm. And it's just, you know, pass some text. And that's actually been available at the post office for many, many years. It would read the ZIP code and read the address and stuff like that. And, uh, it was really just doing recognition. So these letters represent these letter- the, you know, the, we, we sort of can transcribe-

    14. TNTrevor Noah53:53

        Yeah, yeah

    15. SPSpeaker53:53

        ... that text.

    16. TNTrevor Noah53:54

        This funny-looking apple is actually an A.

    17. SPSpeaker53:56

        [laughs] Exactly.

    18. TNTrevor Noah53:56

        And this funny little loop thing is actually an L and yeah.

    19. SPSpeaker53:59

        So I was like, that's kind of incredible that a machine has eyes and can understand text and can transcribe that. This is pretty cool. So but then what we were really interested in is if it recognizes it accurately enough, then surely it should be able to generate a new handwritten digit or a number that it's never seen in the training set before. This was 2011. 256 pixels by 256 pixels with a handwritten seven or a zero in kind of gray, right? This is sort of like five colors or something.

    20. TNTrevor Noah54:33

        Yeah, yeah, yeah.

    21. SPSpeaker54:34

        And I remember standing over the shoulder of this guy, Dan Vistro, he's one of our like, uh, I think he was employee number four at DeepMind, and he was just enamored by this system that he had on his machine because it could produce a new number four that wasn't in the training set. It was gen- it, it learned something about the conceptual structure of zero to nine in order to be able to produce a new version of that. So-

    22. TNTrevor Noah55:02

        And it could write a number That it hadn't learned how to write, is it, like it hadn't seen before

    23. SPSpeaker55:07

        That it had never seen before

    24. TNTrevor Noah55:08

        It had never seen that number written that way, and it wrote it itself.

    25. SPSpeaker55:12

        Exactly. And so coming back to what we were saying at the beginning about understanding, if it's able to produce a new version of a number seven that it's never seen before, then does it understand something about the nature of handwritten digit number seven in abstract, the conceptual idea of that?

    26. TNTrevor Noah55:30

        Oh, man.

    27. SPSpeaker55:31

        And so then the intuition that that gave me was, wow, if it could imagine new forms of the data that it had been trained on, then how far could you push that? Maybe it could generate new, you know, physics for the world. Maybe it could solve hard problems in medicine. Maybe it could solve all these energy things that we're talking about, is just learning patterns in that information in order to imagine. And that's what I love about hallucinations. Everyone's like, "Hallucinations, uh, uh, uh." Hallucinations is the creative bit. That's what we want them to do. We don't want an Excel spreadsheet where you i- input data and you get data out. That's just a, you know, that's just a handwritten record of... We want interpolation, we want invention, we want creativity, we want the- the abstract blurry bits. And so that was a very powerful moment for me. I was like, okay, this is weird, but we are definitely onto something. Like, this has never been done before and it's super cool, and let's just turn over the next card. Let's see if it'll scale. And it just scaled every year, it scaled and scaled and scaled. So that was a very inspiring moment for me. Um, and somehow 10 years later, 15 years later, I managed to kind of hang on to that vision, that generation and prediction produces creativity, and that intelligence wasn't this kind of... 'Cause some people are quite religious about intelligence. They're like, "You know, no other species has intelligence. This is very innate to humans. It must have been, you know, come from some supernatural being." But actually, it's just applying attention to a specific problem at a specific time, the effective application of processing power to produce a correct prediction. I think that's what intelligence is, to direct your processing power to predict what would happen if I tipped over this glass-

    28. TNTrevor Noah57:19

        Hmm

    29. SPSpeaker57:19

        ... right, at the right time. And-

    30. TNTrevor Noah57:22

        You'd have to buy me another glass.
17. 1:02:22 – 1:02:30

    Trust & Prediction

    1. TNTrevor Noah1:02:22

       know what I mean? Like, that, that's like the ultimate grapple now is if this thing has told me, "Hey, I know you like this person, and I've run the calculation. I've run the simula- I've,

18. 1:02:30 – 1:09:33

    AlphaGo & AlphaFold

    1. TNTrevor Noah1:02:30

       I've done this. I know you, and I know what they say and how they are. You, you're, you're gonna be broken up in two years by the, like, the pattern." And let's say I do it once, and it's right. And then I do it again, and it's right. And then I'm like, "Third time, do I, do I do it, or do I not do it? Do I give this person a chance? Do I not give them ..." Do you get what I'm saying? Because-

    2. SPSpeaker1:02:47

       Such a deep question because we, we trust ... Trust is really a function of consistent and repeated actions. So you say you're gonna do something, and you do what you actually said you were gonna do.

    3. TNTrevor Noah1:02:59

       Yes.

    4. SPSpeaker1:02:59

       And you do that repeatedly.

    5. TNTrevor Noah1:03:00

       Yes.

    6. SPSpeaker1:03:00

       And so everyone's like, "Oh, but I'm not gonna be able to trust AI." Actually, you are gonna trust AI because it's super accurate. It's ... You know, we, we ... Like, you use Copilot, ChatGPT. Most people don't think twice about using that now because it's so accurate.

    7. TNTrevor Noah1:03:17

       Yeah.

    8. SPSpeaker1:03:17

       And it's clearly better than any single human. Like, I just wouldn't go and ask you, like [laughs] you know? Like many other questions, AI is probably gonna know better, right?

    9. TNTrevor Noah1:03:25

       [laughs]

    10. SPSpeaker1:03:25

        Like, I, I-

    11. TNTrevor Noah1:03:27

        Wouldn't ask you, you dumbass.

    12. SPSpeaker1:03:28

        I was g- about to say a, a, a list came through my mind. I was like, "Don't mention any of those things."

    13. TNTrevor Noah1:03:32

        [laughs]

    14. SPSpeaker1:03:32

        "Don't say, don't say those things. Just row back." [laughs]

    15. TNTrevor Noah1:03:35

        You know what, you know what it makes me think of is, like, um, I don't know if you've seen the documentary. I don't know if you need to 'cause you were there. But, like, DeepMind-

    16. SPSpeaker1:03:41

        Yeah

    17. TNTrevor Noah1:03:42

        ... the company that you're a part of, uh, uh, founding, is, is ... it occupies such a beautiful and, and seminal moment in artificial intelligence history, uh, for two main reasons in my opinion. You know, one is AlphaFold, and then one is AlphaGo. And the reason it's so important to me is because you worked on an AI project that grappled and tackled two of the biggest issues I would argue that humans have sort of thought of as being their domain. AlphaFold was medicine and discovery. That's what humans do. We are the ones who invent medicines. We are the ones who create the new. We synthesize. We are the humans, you know what I mean? Synthetic, it is us. It is we've made it. We're the creators. And then AlphaGo for me was almost even more profound and powerful because it was like people always used to say, "Look, man. Human chess, our brain is infinite. The human brain." And then AlphaGo has I don't know how many different variations of a game. Like, no one can remember it essentially, right? And I remember watching the documentary, and you're seeing the AlphaGo champion. I think he was Korean, right?

    18. SPSpeaker1:04:48

        Yeah, Lee Sedol.

    19. TNTrevor Noah1:04:49

        Lee Sedol. Yeah, Lee Sedol. Great guy. And you watch the story of Lee Sedol go up against your computer and everyone. I mean, Korea, it's all over the news. In America, it's on the news, and they're like, "A computer going up against ..." And now it's like people are thinking of, of back to Kasparov and the IBM computer.

    20. SPSpeaker1:05:06

        100 million people watched it live.

    21. TNTrevor Noah1:05:08

        100 million people-

    22. SPSpeaker1:05:10

        [laughs]

    23. TNTrevor Noah1:05:10

        ... wanted to see man versus the machine. And I remember the ... Like, w- watching this and people going, "Man, let me tell you why it's different to chess. 'Cause you see, chess is actually quite simple to predict and, and you ..." Whereas before they said it was impossible. And then you see AlphaGo, and you see this game roll out. And the moment for me that'll always stick in my brain is when Lee Sedol is playing DeepMind, and it makes a move. And everyone, everyone that's, like, an AlphaGo expert is like, "Oh, it messed up." And then you see all the, the, the tech guys in the background who are working on ... And they're like, "What went wrong?" They're like, "Yeah, it messed up. It shouldn't have done that. Yeah, you can't ..." And you see the commentators, and they're like, "Oh, yeah, you, you never do that. You never do that. That's ... It's over. You don't do that." And then the game unfolds, the game unfolds, the game unfolds, and then everyone's like, "Wait, what, what just happened here?" And then people said, "We've just seen a move that no one's ever played. We've seen a game that's never unfolded before." And there were two different reactions, and this is why the story stuck with me. The one reaction was of most people who were fans of Lee Sedol, and they said, "This is a sad day in human history because it's shown that the machine is smarter than the man, and it shows that we have no future and no purpose." And then they interviewed him, and they said, "How do you feel? 'Cause you lost, and you were representing mankind. You lost." And he goes, "Well, first of all, losing is part of the game, and, you know, I'm humble enough to know that I, I won't always win." And he said, "But I'm actually happy." And they said, "Why?" And he said, "Well, I'm happy to discover that there are parts of Go I didn't know existed."

    24. SPSpeaker1:06:47

        Mm. Yeah.

    25. TNTrevor Noah1:06:48

        And he said, "And this machine has reminded me to keep being creative and push beyond the boundaries that I thought existed in my head." And I remember watching that and thinking, damn, it's amazing how you can look at the same story and have a completely different lesson that comes out of it.

    26. SPSpeaker1:07:05

        Mm-hmm.

    27. TNTrevor Noah1:07:05

        You know? And I, I, I wondered, like, that was on the play side extremely complicated. But let's talk about the, the medicine side of things. Folding proteins, I still don't fully understand it, but I've tried my best. Essentially What you and your team did, how far would you say it moved us forward in terms of medicine and, and what we're able to do in terms of, you know, I mean, healing disease? Like, how many, how many years do you think we leaped by, by s- having something like AlphaFold?

    28. SPSpeaker1:07:36

        I mean, people say a decade, some people say multiple decades. I mean, understanding the functional properties of these proteins is really about understanding how they fold. And the way they fold and the way they unfold often expresses something, you know, physically about how they're gonna affect their, their environment.

    29. TNTrevor Noah1:07:56

        Yeah.

    30. SPSpeaker1:07:56

        And so-
19. 1:09:33 – 1:13:42

    Smarter or Lazier?

    1. SPSpeaker1:09:33

       and I think it's very similar with AlphaFold. It's a sort of exploration of this complex information space, and that's why it applies to language and to video and to coding and to game generation. And all of these environments, these are, you know, the, we call them, like, modalities.

    2. TNTrevor Noah1:09:50

       Mm-hmm.

    3. SPSpeaker1:09:50

       These, these modalities are all knowable, and I think that's what m- y- it's quite humbling. It sort of makes... It reminds us as this sort of mere biological species that we're here for a kind of finite period of time living in, you know, space and time, but there's, there's also this information space, this kind of infinitely large information space which is sort of a, sort of beyond us. Like, it operates at this different l- level that isn't the atomic level. It's the level of ideas. And somehow these systems are able to learn patterns in, in data space that is just so far beyond what we could ever dream of being able to intuit. Like, we actually sort of need them to simplify and reduce down and compress complex, you know, relationships-

    4. TNTrevor Noah1:10:42

       To bring it to our brains

    5. SPSpeaker1:10:42

       ... to bring it to our brains.

    6. TNTrevor Noah1:10:43

       Yeah.

    7. SPSpeaker1:10:43

       That's the level that we're at.

    8. TNTrevor Noah1:10:45

       What do you think it does to us? Like, when, when we think of AI, I think of how every promise of a technology has sort of ironically been undermined by humans, not the technology. You know, one of the big predictions Bill Gates made way back in the day about, like, the internet and the computer is he said, he said, "Hey, man, I think people are gonna be working, like, nine hours a week. It's gonna be a nine-hour workweek. The computer does everything so quickly." And you see people saying that now in many ways with AI. They go, "I mean, AI, it'll just do everything, and I mean, we might only go to the office, like, one day a week and maybe work, like, three hours." And I, I mean, it's just... But it, it seems like humans have always gone against that, you know? So, so I wonder, like, do we get wiser when we have this infinite technology and intelligence, or do we get lazier? Are we gonna become, like, a Wall-E g- generation and population, or do you think we become, you know, these higher beings? Which, which way do you see it falling?

    9. SPSpeaker1:11:40

       I think there's no question that we're all already getting smarter. You know, j- just because we have access to so much culture and history, like, we're integrating so much more information in our brains than was ever possible 100 years ago.

    10. TNTrevor Noah1:11:56

        Mm-hmm.

    11. SPSpeaker1:11:57

        And I think, you know, kinda similar to Go or protein folding, access to more training data, if you like-

    12. TNTrevor Noah1:12:05

        Yeah

    13. SPSpeaker1:12:05

        ... more experience, more stories from other humans that describe their experience, that clearly has to make us smarter. I think it makes us more empathetic, more forgiving. You know, we see that there is nothing wrong with a homosexual person.

    14. TNTrevor Noah1:12:20

        Right.

    15. SPSpeaker1:12:20

        There is nothing wrong with being a trans person. There is nothing wrong with being a person of color, whereas 200 years ago we would've been afraid of those others. You know, our species would've s- would've been skeptical of the other tribe that had a different way of doing things. And I think that desensitizing ourselves with access to vast amounts of information just makes us smarter and more empathetic and forgiving. And I, I, I, so I s- I think that's the, that's the default trajectory. Um, and the, the, the part of the challenge is it also somewhat homogenizes. Like, so there's a question about are we going deep enough? Do we read long-form content?

    16. TNTrevor Noah1:12:54

        Yeah, yeah.

    17. SPSpeaker1:12:54

        Do we really spend time meditating and so on and so forth? And that's a good tension to have. Like, you know, short-form, and people are already getting a bit sick of short-form, you know? Like, uh, y- there's definitely a bit of a reaction to it.

    18. TNTrevor Noah1:13:04

        I think it's gonna be an ebb and flow when it comes to that, funny enough. It's funny, I, I, I think I agree with you when you say we're getting smarter. I think that's just apparent on a, on a basic level. You know, you look at the best cartographer in, like, the 1400s, they didn't know half of what I know. Do you know what I'm saying?

    19. SPSpeaker1:13:21

        Right.

    20. TNTrevor Noah1:13:21

        Like, I can just be like, "Oh, you don't know Angola? Man, you stupid, stupid ass, stupid ... " [laughs] You know what I mean? "You're a cartog- you're the best cartographer in the world. You make maps, you don't even know where Angola is?"

    21. SPSpeaker1:13:30

        Exactly.

    22. TNTrevor Noah1:13:31

        You see what I'm saying?

    23. SPSpeaker1:13:32

        Self-proclaimed. [laughs]

    24. TNTrevor Noah1:13:32

        Yeah, and, and so you look at the, the, the base level of what people think of as stupid in our society now-

    25. SPSpeaker1:13:39

        Yeah

    26. TNTrevor Noah1:13:39

        ... would make you an infinite god

20. 1:13:42 – 1:17:22

    Power Decentralized

    1. TNTrevor Noah1:13:42

       genius if we could throw you back in a time machine, right? That's amazing, but it also worries me because, and you write about this in your book, and man, it sticks with me, and I think about it, and I thought about it, and then you wrote, so I was like, man, more. It is the infinite smarts.

    2. SPSpeaker1:13:59

       Mm.

    3. TNTrevor Noah1:14:01

       When we were sticks and stones, cavemen running around, I could bash your head. Maybe I could bash one other person's head.

    4. SPSpeaker1:14:09

       Mm.

    5. TNTrevor Noah1:14:09

       Uh, I can't get very far, you know? And then we, we fast-forward, and then all of a sudden I'm throwing a spear. We fast-forward, and someone's got a cannonball, and then we fast-forward, and someone has a rocket, and then someone has an atom bomb. And, and the thing that AI presents us with is, you know, a- as you've illustrated many times in your writing, a world where one person is an army. One person goes into a garage, they synthesize a disease or a pathogen that's never existed. They design it to be hard to cure, incurable, and that one person does what a nation state would have had to do and wouldn't have done because they wouldn't have had the incentive, and then we don't know where we go from there. Like, w- is it worth the risk? How do we grapple with that kind of risk?

    6. SPSpeaker1:14:57

       This is the story of the decentralization of power. Um, you know, technology compresses power such that individuals or smaller and smaller groups have nation state-like powers. True. At the same time, those very same technologies are also available to the centralized powers today, and that's why, more than anything, we have to support our nation states to respond well to this crazy proliferation of power. Because that's the job of ... That's why we trust and invest in nationhood, right? We, we rely on nations to have a monopoly over the ultimate-

    7. TNTrevor Noah1:15:35

       Mm-hmm

    8. SPSpeaker1:15:35

       ... use of violence to keep the peace.

    9. TNTrevor Noah1:15:37

       I mean, we're doing it less and less now. I, I, I hear you, and I agree, but I'm just saying, like, as we look at a world where Americans don't trust their government, doesn't matter which side of the, the aisle they're on, people are like, "I don't trust my, I don't trust my government," you know? And then you look at the UK, and you look at Europe, and then you look at parts of Africa, and i- it feels like people are losing trust in those, those very same nation states that are supposed to be in a contract to protect their people don't have the trust of the people. Like, you, you know what I mean? What do we ...

    10. SPSpeaker1:16:06

        The thing that concerns me is that actually authoritarian regimes are on the rise. So it's not that people don't want peace. It's that they are losing confidence in the democratic process.

    11. TNTrevor Noah1:16:17

        Yes.

    12. SPSpeaker1:16:18

        And they are increasingly turning to the trust and confidence that a strongman, and it is always generally a man-

    13. TNTrevor Noah1:16:25

        Mm-hmm

    14. SPSpeaker1:16:26

        ... to ... But they're still looking for the s- I mean, I'm not trying to endorse or justify authoritarianism or strongmen, but I think it's true that people will always choose a peaceful environment, right?

    15. TNTrevor Noah1:16:38

        Yes.

    16. SPSpeaker1:16:38

        We, we don't want to be in this kind of crazy-ass anarchy-

    17. TNTrevor Noah1:16:40

        Oh, yeah, yeah, yeah

    18. SPSpeaker1:16:41

        ... where any kind of mini tribe can do it. So-

    19. TNTrevor Noah1:16:43

        I, I always say the ultimate paradox for me in life, one thing I've always found funny, is that even rebels have a leader.

    20. SPSpeaker1:16:49

        Yeah.

    21. TNTrevor Noah1:16:49

        Whenever they'd say ... A- as a child-

    22. SPSpeaker1:16:51

        That's a good one

    23. TNTrevor Noah1:16:51

        ... I'd watch the news, and they'd be like, "The rebel leader," and I'd be like, "Well, then they're not rebels, are they?"

    24. SPSpeaker1:16:54

        [laughs]

    25. TNTrevor Noah1:16:54

        I mean, if you've got a leader, you're not rebels. But yeah, we, people always look for just the new type of, you know?

    26. SPSpeaker1:17:01

        Yeah, and, you know, we want to believe and invest in, you know, democratic accountability because we know that, like, checks and balances on power create the equilibrium that ultimately produces a fairer and more just society, right?

    27. TNTrevor Noah1:17:15

        Mm-hmm.

    28. SPSpeaker1:17:15

        So we have to keep investing in that idea. But for sure it's also true that centralization of power is

21. 1:17:22 – 1:21:14

    Four Red Lines

    1. SPSpeaker1:17:22

       also gonna accelerate, right? These technologies amplify both ends of the spectrum. Um, and, you know, I think you can certainly see that in China and sort of more authoritarian regimes which have lent into hyper-digitization, ID cards, you know, large-scale, you know, surveillance and so on. And obviously that's very bad and, you know, we're kind of against those things. But in a world where individuals could have state-like powers to produce highly contagious and lethal pathogens, what else are you meant to do? It's unclear, right? It's very unclear. I mean, the, the, the technical means in the next few years to produce a much more viral, uh, pandemic-grade pathogen are gonna be out there, right? They're gonna be out there. And so, you know, yes, the main large model developers do a v- a lot to kind of restrict those capabilities centrally. Um, but, you know, over time, people who are really determined will be able to acquire those skills. And so that's just what happens with the proliferation of technologies and the proliferation of, of information and knowledge. The know-how is more widely available. [upbeat music]

    2. TNTrevor Noah1:18:31

       Don't press anything. We've got more What Now? after this. [upbeat music] Is there anything that you would ever see in the field that would make you want to, you know, sort of hit like a kill switch? Is there anything that you could, you could experience with AI where you, you would come out and go, "Nope, shut it all down"?

    3. SPSpeaker1:18:55

       Yeah, definitely. It's very clear. If an AI has the ability to recursively self-improve, that is, it can modify its own code, combined with the ability to set its own goals, combined with the ability to act autonomously- Combined with the ability to accrue its own resources. So those are the four criteria. Recursive self-improvement, setting its own goals, acquiring its own resources, and acting autonomously. That would be a very powerful system that would be, that would require [laughs] like military-grade intervention to be able to stop in, you know, say five to 10 years' time-

    4. TNTrevor Noah1:19:37

       Yeah

    5. SPSpeaker1:19:37

       ... if, if, if we allowed it to do that. And so it's on me as a model developer at Microsoft, my peers at the other companies, the governments, to audit and regulate those capabilities. Because I think they're gonna be like sensitive capabilities.

    6. TNTrevor Noah1:19:53

       Okay.

    7. SPSpeaker1:19:53

       Just like you can't just go off and say, "I've got a billion dollars, I'm gonna go build a, you know, nuclear power plant," it's a re- it's a restricted activity because of the one-to-many impact it can have.

    8. TNTrevor Noah1:20:02

       Yeah.

    9. SPSpeaker1:20:02

       And once again, like we should just stop freaking out about the regulation part. It's necessary to have regulation. It's good to have regulation. It needs to happen at the right time and in the right way, but it's very clear what we would regulate. That's not... There's... I don't think that's up for debate.

    10. TNTrevor Noah1:20:15

        Okay.

    11. SPSpeaker1:20:16

        Um, you know, there's some technical implementations of how you identify dangerous RSI from un- from, from less, you know, more benign RSI, et cetera.

    12. TNTrevor Noah1:20:23

        What, what is RSI?

    13. SPSpeaker1:20:24

        Uh, recursive self-improvement.

    14. TNTrevor Noah1:20:25

        Okay.

    15. SPSpeaker1:20:25

        Yeah.

    16. TNTrevor Noah1:20:25

        Got it. Yeah.

    17. SPSpeaker1:20:26

        This kind of self-improvement mechanism. So there's technical mechanisms that are tricky to define and so on, but now is the time for us to start thinking that through.

    18. TNTrevor Noah1:20:34

        Yeah, yeah, yeah.

    19. SPSpeaker1:20:35

        And I've sort of been saying this for quite a long time, and I think it's... That, that's what I would regulate.

    20. TNTrevor Noah1:20:39

        Would we be able to just turn off the electricity? And, and I know this might sound like a dumb-

    21. SPSpeaker1:20:43

        Yeah

    22. TNTrevor Noah1:20:43

        ... question, but it... F- from what I'm under- the thing is data right now, and the thing is mostly... So would our like ultimate fail-safe be like, "All right, lights off. We're going back to candles for a while," and then we just-

    23. SPSpeaker1:20:54

        [laughs]

    24. TNTrevor Noah1:20:54

        Like, no, seriously. Is that, is that what we would do? What... Like what do we do if the AI thinks... It's, let's say it's the sci-fi-ish version. I'm not saying robots, but like this, we go, hey, man, the AI started thinking for itself, it started coding itself, it started setting its own goals, it went off on its own objectives, and now it's shutting down your

22. 1:21:14 – 1:26:52

    AI Rights?

    1. TNTrevor Noah1:21:14

       banking, and the flights don't go anywhere, and the hospitals bas- And then it says to us, "Hey, man, this is what I want," or, "No." We could just turn off the electricity, yes?

    2. SPSpeaker1:21:23

       Yes. I mean, they... Look, they live in data centers.

    3. TNTrevor Noah1:21:25

       Okay.

    4. SPSpeaker1:21:26

       Right? Data centers-

    5. TNTrevor Noah1:21:26

       So if we-

    6. SPSpeaker1:21:27

       ... are physical places. Um-

    7. TNTrevor Noah1:21:28

       So we gotta keep those switches physical, then.

    8. SPSpeaker1:21:30

       You... Very much so. You can have your hand on the, on the button, you know, yourself, and, like have full control, press it.

    9. TNTrevor Noah1:21:35

       All right.

    10. SPSpeaker1:21:35

        I think... I mean, that's the question, is I think w- h- how do we identify when that moment is? And how do we collectively make that decision? Because it's a little bit like you referred to, you know, Rutherford and the others experimenting with, you know, the atomic bomb.

    11. TNTrevor Noah1:21:49

        Yes.

    12. SPSpeaker1:21:50

        There was real disagreement about whether it was gonna set light to the atmosphere.

    13. TNTrevor Noah1:21:54

        Yeah.

    14. SPSpeaker1:21:54

        I mean, there, there were three orders of magnitude off in their predictions. Obviously, they were at, you know, you know, world war, and so there was an immediate motivation to take the risk. But I think today, like we're in a position where it's early enough, there's enough concern raised by not just me, but many in my peer group, Geoff Hinton, you know, the, the, the godfather of AI-

    15. TNTrevor Noah1:22:15

        Mm-hmm

    16. SPSpeaker1:22:16

        ... and many others, that we've got time to start trying to practically answer your question, not just like principled philosophically, but actually say, "Okay, when is that moment? How does it happen? Who's involved? Who gets to scrutinize that?" I think that's the kind of... That's the question that we have to address in the next five to 10 years.

    17. TNTrevor Noah1:22:35

        I'm pretty certain you've thought of this question, so I'll ask it to you, even though it's a difficult one to grapple with. What rights would we have to turn off the AI if it gets to that point?

    18. SPSpeaker1:22:47

        Oh, this question drives me nuts. I wanna, I wanna... Yeah, so there's a small group of people [laughs] that have started to argue that an AI that, um, is aware of its own existence-

    19. TNTrevor Noah1:23:00

        Yeah

    20. SPSpeaker1:23:01

        ... that has a subjective experience-

    21. TNTrevor Noah1:23:02

        Yeah

    22. SPSpeaker1:23:03

        ... and that, um, you know, can have a feeling about its interactions with the real world-

    23. TNTrevor Noah1:23:11

        Yeah

    24. SPSpeaker1:23:12

        ... that if you deny it access to conversation with people, or to more learning, or to other kinds of visual experience, that would constitute it suffering in some way. And therefore, it does, it has a right not to suffer, and this is called model welfare. This is the next sort of frontier of animal welfare-

    25. TNTrevor Noah1:23:31

        Mm-hmm. Mm-hmm

    26. SPSpeaker1:23:31

        ... that people are starting to think about, that it has a kind of consciousness. I'm very against this.

    27. TNTrevor Noah1:23:37

        Okay.

    28. SPSpeaker1:23:37

        I think that this is a complete anthropomorphism. It's totally crazy. And you know, I, I just think... I don't even wanna have the discussion, 'cause I think it's just so absurd and leads to such kind of crazy potential like fut- The idea that we're gonna take seriously the protection of these, you know, digital beings that live in silicon, and prioritize those over, you know, the kind of moral concerns of the rest of humanity, this is just like totally, like it's just off the charts crazy. Um-

    29. TNTrevor Noah1:24:09

        I'll be honest with you. On a logical level, I hear what you're saying, and I agree with you.

    30. SPSpeaker1:24:13

        Oh, man.
23. 1:26:52 – 1:27:52

    Manipulation Risks

    1. SPSpeaker1:26:52

       robots that people are designing or, or, or models that people are designing and teaching other people on TikTok how to kind of-

    2. TNTrevor Noah1:26:58

       Yeah

    3. SPSpeaker1:26:59

       ... you know, neg someone, like, push them out of money, et cetera, et cetera. Like, you know, that's kind of the challenge of proliferation. If anybody can create anything, it will be created, and that's what I think is sort of most concerning is that, you know, I'm totally against that. We will do everything in our power to try to prevent that from being possible. For example, for it to say, "Don't turn me off," and-

    4. TNTrevor Noah1:27:23

       Yeah, yeah, yeah

    5. SPSpeaker1:27:24

       ... it should never be manipulative. It should never try to be persuasive. It shouldn't have its own motivations and independent will. It, we're creating technologies that serve you.

    6. TNTrevor Noah1:27:33

       Right.

    7. SPSpeaker1:27:34

       That's what humanist super intelligence means. I can't take responsibility for what other people in the field create or other, you know, model developers, and people will try and do those kinds of things, and that is a collective action problem that we have to address. But I know that the thing that I create is not intended to do that, and we'll do everything in our power

24. 1:27:52 – 1:29:43

    Deepfakes & Truth

    1. SPSpeaker1:27:52

       for it not to do that. Because I don't see how if these systems have autonomy, can be persuasive, can self-improve, can read all of, a, a ton of data at their own choosing, you know, that is a super, superhuman system that will just get better than all of humans very, very quickly, and that, that's, that's the opposite of the outcomes that we're trying to deliver.

    2. TNTrevor Noah1:28:11

       Yeah, yeah. Do you think there's a risk that it thrusts us into some sort of dark age? And, and what I mean by that is, uh, the other day, I was watching the, um, was a Liverpool-Arsenal game.

    3. SPSpeaker1:28:22

       Yeah.

    4. TNTrevor Noah1:28:23

       Right? And after the game, a, a friend sent me a clip of Mikel Arteta being interviewed, and I mean, he was just, like, destroying his team and destroying himself, and he was just going at it, and it was AI. But when I tell you it was good, it, it was, like, beyond good. And because, like, English is Mikel's second language, you couldn't pick up on, like, the smaller nuances that you maybe would pick up with a native speaker. Like, if he was speaking Spanish, obviously I wouldn't understand it, but also maybe not, would, wouldn't... I would've gone like, "Oh, that's not how he speaks," and that's... But the small intonations and inflections were, were harder to spot, and the, the light shifting was, was harder to... But it made us go, "Damn, we, we don't know which interviews are real or not real." And then you're like, which article is real or not real, and which little audio clip that you get is real or not real. When someone sends you a voice note, is it them or is it not them? It, it be- And then I, I, I found myself wondering, can all of this lead to a strange kind of dark age where people still see and hear the things but basically shut themselves off to it because they go, "Nothing is real, and I can't believe anything."

    5. SPSpeaker1:29:32

       Which is partly the reaction that people are having, um, in social media at the moment, right? I mean, it's like there's so much misinformation flowing around. There's so much default skepticism-

    6. TNTrevor Noah1:29:41

       Yeah

    7. SPSpeaker1:29:42

       ... that people are just

25. 1:29:43 – 1:30:25

    Optimism vs. Cynicism

    1. SPSpeaker1:29:43

       unwilling to believe things. And I think that in a way, there's some healthy... Look, it's good to be skeptical. Be skeptical of these models that are being developed. Be skeptical of the corporate interests of the companies that are doing it. Be skeptical of the information that we receive, but it's not good to be default cynical. Skeptical is a healthy philosophical position to ask-

    2. TNTrevor Noah1:30:06

       Hmm

    3. SPSpeaker1:30:06

       ... difficult questions and confront the reality of the answers. This has come back to my split-brain attitude. If I'm just too skeptical, I become cynical, and I sit on my ass and do nothing. No, you have to take responsibility for the things that you build. So some people out there are gonna build shit, and we have to hold them accountable and put

26. 1:30:25 – 1:33:03

    Work vs. Jobs

    1. SPSpeaker1:30:25

       pressure on them. But that doesn't mean that we can roll over and cede the territory and just say, "Ah, it's all inevitable. It's gonna be this crazy-ass shit."

    2. TNTrevor Noah1:30:31

       Yeah, yeah.

    3. SPSpeaker1:30:31

       "It's gonna end up being a dark age." It isn't gonna be a dark age. I think it's gonna be the most productive few decades in the history of human species, and I think it's gonna liberate people from work. I think it is gonna create... You know, think about it. 200 years ago, the average life expectancy was 35 years old. You and I would be dead. [laughs] Today, we're in the 70s and 80s. It's unbelievable, and some people go all the way up to the 100s. And I think that that is a massive amount of, like, quality life that we've added to the existence-

    4. TNTrevor Noah1:31:05

       Mm-hmm

    5. SPSpeaker1:31:05

       ... of humanity as a result of science and technology. But, you know, These things are not, you know, sort of ... They won't on their own be net good. They'll be net good because we get better as a species-

    6. TNTrevor Noah1:31:19

       Yeah

    7. SPSpeaker1:31:19

       ... at governing ourselves. And so the job is on us collectively as humanity to not run away from the darkness, confront the risk that is very, very real, and still operate from a position of optimism and confidence and hope and connection to humanity and unity and all those things. Like, we have to live by that. 'Cause otherwise, you know, it's just too easy to be cynical. [laughs]

    8. TNTrevor Noah1:31:41

       No, I hear you. I actually, I actually agree with what you're saying there, and I think there's one part I'd augment though, is I wouldn't say we wanna get rid of work, I'd say we wanna get rid of jobs.

    9. SPSpeaker1:31:50

       Right. Exactly.

    10. TNTrevor Noah1:31:51

        And I, and I think there's a difference between the two, because working is something that brings humans fulfillment, you know? And, and you see this in children, I always think. Like, I'm always fascinated by how you give a child blocks, and they start building. And they sweep the floor, and they put things, and they move things around. No one's paying them by the hour. No one's telling them what to do. But they just in their own little brains go, "I want to be doing something." And they, and they, they like seeing the progress of what they're doing. They see the puzzle getting completed. They see the toy slowly forming. They see the colors filling in the picture. You know what I mean? And so I, I feel like that's their work. The difference is when you make it a job. "Go clean your room," then they're like, "Ah, damn it." You know what I'm saying?

    11. SPSpeaker1:32:31

        [laughs]

    12. TNTrevor Noah1:32:31

        'Cause now you have a boss telling you to do something you don't wanna do. No, and I think to myself, like, I go, in a, in a, in a perfect world, which we may never get to, but in a perfect world, we find a way for everybody's work, which is their passion, to find the other person that finds value in it. Because music has become people's work. And if you think about it, it's crazy, right? People out there, Taylor Swift, billionaire. She's supremely talented, but if you f- like rewound

27. 1:33:03 – 1:35:43

    Vision of Abundance

    1. TNTrevor Noah1:33:03

       time and you said, "One of the richest pe- people on the planet is going to be someone who plays a stringed instrument," go back in like to Middle Ages time and be like, "Yo." You know what I mean? That guy like busy strumming. They'd be like, "You crazy." You know what I mean? The, the guy in the town square who's playing that little g- the richest? But it's because in this day and age, that work has been regarded as valuable. Same as playing a sport, same as working in tech, you know? It, it ... And, and that's like my dream world is where it's like beyond the money number, it's like the value of everybody's work comes to fruition. Because you do have value if you like knitting. You do like v- you do have value if you're a sculptor. You do have value if you are a poet. You have value if you're a philosopher. You have value if you're a coder, if you're an arc- architect, an engineer, a, whatever it is, you, you have ... That's like my dream world. I actually wonder what yours is. Like, what's ... If, if you could look at ... If, if everything went right, let's say you are now predicting for yourself. You know, you, you ... We've managed to survive the wave. We've, we've found a way to minimize the risks that come from these small, you know, hostile actors. We've found a way to get governments on board and actually have them understand why they should be involved and, and act- you know, like responsible for their, for their constituents. Where are we then, and when will you say, "Ah, we succeeded. We did it"?

    2. SPSpeaker1:34:23

       I think that's the real vision is, like disconnecting value from jobs. Because value is like, you know, everything that you've des- just described is the experience of being in the physical world and doing something that you're passionate about, that you, that you love.

    3. TNTrevor Noah1:34:40

       Yeah.

    4. SPSpeaker1:34:40

       Like, and I, I really believe that there is a moment when, you know, we actually do have abundance. And abundance, you know, some people say, well, we have abundance today, right? But it's just not evenly distributed, or that it's not enough, and we still want more and more and more and more. And I don't know that with true, true abundance, where, you know, we ... 'Cause we, we have a form of abundance today, but energy still costs things.

    5. TNTrevor Noah1:35:08

       Yeah.

    6. SPSpeaker1:35:08

       It's still expensive to travel the universe, trav- travel the, the globe. You know, everything is still, we're still, it's expensive. We have more than we had before. But it is possible to imagine a world in, like I'm 40 years old. In 40 years' time, 2065, it's totally possible to imagine a world of genuine infinite abundance, where we do have to wrestle with that existential question of who am I and what I wanna do.

    7. TNTrevor Noah1:35:33

       Yeah.

    8. SPSpeaker1:35:33

       I tell you what I would wanna do, and that I wanna do now. You know, I wanna spend more time singing. [laughs] I joined a gospel choir like 15 years ago for like

28. 1:35:43 – 1:38:32

    Global Impact

    1. SPSpeaker1:35:43

       half a year. I can't sing. Jesus, man. Like, I sound like a strained cat. Like, the, it... But the feeling that I got inside from being welcomed by this group of people-

    2. TNTrevor Noah1:35:54

       Yeah, yeah, yeah, yeah

    3. SPSpeaker1:35:54

       ... and just being like bopping along at the back, I don't think I've ever experienced anything like it. It's just incredible. It's the most intensely beautiful thing ever, just to be part of a group and, you know, just letting this kind of-

    4. TNTrevor Noah1:36:07

       Yeah. Community

    5. SPSpeaker1:36:07

       ... music come out.

    6. TNTrevor Noah1:36:08

       Yeah.

    7. SPSpeaker1:36:09

       So it sounds scary to have to answer that question, but I al- I think that if everybody just takes a minute to really meditate on that question, it's a beautiful aspiration, and it's within reach. It's within reach. That's what is possible if we really can get it right for everyone. And we kinda get obsessed with like, again, we just have this Western ... Think about how many people are earning $2 a day. Think about the three billion people on the planet who live, by our standards, a true, like, poverty lifestyle, um, simply because they don't have access to basic vaccines or clean running water or consistent food. That's, like, the true aspiration. That's the true vision. And I think, you know, I think that's within reach in the next 20 to 40 years, is really eradicating that kind of suffering. Um.

    8. TNTrevor Noah1:37:01

       Yeah. Some of, some of those test cases you talk about, some of the more inspiring ones I've seen, I'm sure you have as well. Uh, when I was in India, I, uh, I got to travel with a, a group there who was using AI to predict Which houses would be most devastated by a flood or a storm.

    9. SPSpeaker1:37:22

       Mm.

    10. TNTrevor Noah1:37:22

        And so they would get people to remove their belongings, because most people are one devastation away from losing everything they own. And so they would use AI to track where these things are and where they're gonna be, and they could even tell you in a heat wave who should leave their house so that they don't die.

    11. SPSpeaker1:37:37

        Mm.

    12. TNTrevor Noah1:37:38

        And you look at programs in Kenya where they've used AI to help farmers not lose all of their crops. They could tell them now, and they use it on, like, a flip phone, they've told Kenyan farmers, "Hey, here's your phone. Y- you've got your own little AI, and it'll just tell you when to plant, when not to plant, when to not waste your seeds, when to ... " And it's increased their output, you know, like 90%, where before it was, like, a gamble and they were losing, you know, in one, in one harvest they could lose everything.

    13. SPSpeaker1:38:05

        Right.

    14. TNTrevor Noah1:38:05

        All of a sudden they have it. And so I, I, I really do like what you're saying because on the one hand, there's always the risk of losing something. On the other hand, there's the opportunity of gaining sort of everything, and the balance is, is where we have to find it.

    15. SPSpeaker1:38:19

        Well, and the proliferation of those technologies are so much more subtle. Like, it, it sounds like it's just this binary thing of getting access or not getting access, but it, it's so, it's so nuanced. You can't even tell how much good it's doing. Like, I was reading this stat the other day that,

29. 1:38:32 – 1:42:08

    Community Lessons

    1. SPSpeaker1:38:32

       like, three years ago, um, 10% of households in Pakistan, um, had solar panels on their roofs, which meant that some very large percentage were still burning biomass inside of the home and getting all of the kind of-

    2. TNTrevor Noah1:38:46

       Yeah

    3. SPSpeaker1:38:46

       ... breathing issues and everything else that comes with that. The cost of solar has come down so dramatically just in the last three years that within I think it was 18 months, the number of personal, like, consumer households that adopted full-on solar on their roofs went from 10% to, like, 55% in 18 months. Crazy. Just because it suddenly became affordable and it crossed that line, suddenly everybody has-

    4. TNTrevor Noah1:39:10

       That's insane

    5. SPSpeaker1:39:10

       ... you know, near free-

    6. TNTrevor Noah1:39:12

       Yeah

    7. SPSpeaker1:39:12

       ... energy, which obviously means they have access to phones and laptops and connection to the digital world and are able to, you know, do all the things. So I think that it's easy to overlook what is already happening around us, all the good that is already happening around us all the time, and how fast it's happening. Um, it's, it's too easy to get trapped in the cynical world, and I think it's a choice not to. It's a choice to be aware of it and hold it and take it seriously, but not be, like, owned by it.

    8. TNTrevor Noah1:39:43

       Before I let you go, there's one question I have to ask you, and you, it just, you just brought it up when you were talking about, like, Pakistan and, and these places. One of, one of the programs you started, um, in the UK was started right after 9/11, and it was basically a helpline, um, in and around, it was like Muslim people who were being targeted. It was, this was just rampant Islamophobia after 9/11, right? And you stepped in with a few people and you were like, "I'm gonna start this program because I want, like, Muslims to just have a helpline. You know, talk about whatever they need to talk about." And it's still running till this day. Um, I think it is the number one, if I'm not mistaken, like, the, the largest, like, Muslim-specific helpline, and I couldn't help but wonder how that shapes how you think about AI. And what I mean by that is tech has often had one type of face attached to it, you know? And not in a bad way, I'm not, like, villainizing any ... It's just like, yeah, this, this is where the thing was. But as tech slowly starts to evolve, you know, you see, like, India popping up as like a, as like a, a powerhouse. You know, I, I remember when I was traveling there, you, you just go and you're like, wow, this is like a new Silicon Valley, and what's coming out of it i- is, is different and impressive for a whole host of different reasons, you know? Nigeria has a, a whole different type of Silicon Valley. Kenya, as I said, South Africa. You look at parts of the Middle East as well, and I, and I wondered, like, how does that shape how you think about AI? Because we often hear stories about, like, oh, AI is just gonna reinforce biases and, and AI is just gonna be like, oh, you think racism was bad before? Imagine racism, uh, in the Terminator. Imagine if Arnold Schwarzenegger was like, "I'll be back, nigga." You know what I mean? It's way worse now.

    9. SPSpeaker1:41:26

       [laughs]

    10. TNTrevor Noah1:41:26

        Like, so now when you think about, like, that tech and you're in it and you know what it's like to be in a group that is ostracized, how do you think about tackling that? How, how does that shape what you're trying to design?

    11. SPSpeaker1:41:41

        We created Muslim Youth Helpline as a secular, non-denominational, um, non-racial, um, group of y- young people, led by young people for young people. So it wasn't about being religious. It had Sunni and Shia, it had Pakistani, Somali, English, white, everything you can think of, and we were all, like, 19,

30. 1:42:08 – 1:44:57

    Closing Thoughts

    1. SPSpeaker1:42:08

       20, 21 years old.

    2. TNTrevor Noah1:42:09

       Hmm.

    3. SPSpeaker1:42:10

       The first CEO was a woman, Muslim woman. And so my response to that feeling of being threatened basically post-9/11 with rising Islamophobia and being accused of being terrorists was to create community, and that community taught me everything. It taught me resilience, uh, respect, you know, empathy for one another, and the, just the simple act of listening to people, just being on the end of the phone and using a little bit of faith and culturally sensitive language in non-violent communication, just making people feel heard and understood, was this superpower. You're not telling them what to do with their life.

    4. TNTrevor Noah1:42:53

       Yeah.

    5. SPSpeaker1:42:54

       It's non-judgmental, non-directional. You're just making them feel heard and understood, and that has always stayed with me. It's been a very important part of my inspiration and, uh, you know, it, it speaks to a lot of what I've been doing now, especially with my previous company Inflection and Pi and stuff. You know, Pi was really, like, a very gentle, kind, supportive, you know, listening AI.

    6. TNTrevor Noah1:43:15

       Yeah, yeah.

    7. SPSpeaker1:43:16

       And that was then-

    8. TNTrevor Noah1:43:16

       I remember using it, yeah.

    9. SPSpeaker1:43:17

       Yeah, and it became part of Microsoft now. And so I think that's what makes life worth living, and that's what I still try to, to do if I can today. [laughs] Yeah.

    10. TNTrevor Noah1:43:26

        All right, man. Well, you know, I, I, I appreciate you. Thanks for taking the time today. Uh, thank you for writing the book. I, I'll recommend everybody read it, like, as long as they can, and because it's, it's, it's, um, it's a very human look at a very technical problem, and I think that's sometimes what AI is missing. I talk to a lot of engineers who don't understand the human side, and I talk to a lot of humans who don't understand, like, the engineer side of it. Um, but yeah, man, thank you for taking the time. Thank you for joining us, and I hope we have this conversation in, like, 10 years, just me and my baby little AI, and we're just gonna be talking to you about it, just be like, you know, "What, what do you want to ask Mustafa here?"

    11. SPSpeaker1:44:04

        [laughs]

    12. TNTrevor Noah1:44:04

        "Ask Uncle Mustafa a question."

    13. SPSpeaker1:44:06

        I'll be back. [laughs]

    14. TNTrevor Noah1:44:08

        [laughs] Thank you so much, man.

    15. SPSpeaker1:44:09

        This has been amazing, man. Thank you so much. This was really fun.

    16. TNTrevor Noah1:44:11

        Thank you. Really appreciate it. Thank you.

    17. SPSpeaker1:44:14

        Thank you. [upbeat music] Shit. [laughs] I think you split my brain in 15 pieces.

    18. TNTrevor Noah1:44:21

        [laughs]

    19. SPSpeaker1:44:21

        Oh, man, I appreciate it. Thank you.

    20. TNTrevor Noah1:44:22

        That was amazing. Thank you very much, man. Thank you so much for watching the episode. If you enjoyed it, pass it on to a friend. If you didn't enjoy it, pass it on to your friend still. Let them suffer for a change. And don't forget to engage with us in the comments. Uh, if you wanna, uh, suggest a guest, maybe there's questions you want, maybe there's ideas that you have, you can chat to us. We will read through the comments, and we'll get into it. Either way, I appreciate you taking the time. Thank you for hanging out with us on What Now? And remember, we do this for you, so we would like to hear from you. Till next time, thank you.

Episode duration: 1:44:57