Geoffrey Hinton on AI superintelligence and human extinction

1. 0:00 – 2:11

   Intro

   1. SBSteven Bartlett0:00

      They call you the godfather of AI. So what would you be saying to people about their career prospects in a world of super intelligence?

   2. GHGeoffrey Hinton0:07

      Train to be a plumber.

   3. SBSteven Bartlett0:08

      Really?

   4. GHGeoffrey Hinton0:09

      Yeah.

   5. SBSteven Bartlett0:10

      Okay. I'm gonna become a plumber. Geoffrey Hinton is the Nobel Prize-winning pioneer whose groundbreaking work has shaped AI...

   6. NANarrator0:17

      And the future of humanity.

   7. SBSteven Bartlett0:19

      Why do they call you the godfather of AI?

   8. GHGeoffrey Hinton0:21

      Because there weren't many people who believed that we could model AI on the brain so that it learned to do complicated things, like recognize objects in images or even do reasoning, and I pushed that approach for 50 years. And then Google acquired that technology and I worked there for 10 years on something that's now used all the time in AI.

   9. SBSteven Bartlett0:36

      And then you left?

   10. GHGeoffrey Hinton0:37

       Yeah.

   11. SBSteven Bartlett0:38

       Why?

   12. GHGeoffrey Hinton0:38

       So that I could talk freely at a conference.

   13. SBSteven Bartlett0:40

       What did you wanna talk about freely?

   14. GHGeoffrey Hinton0:42

       How dangerous AI could be. I realized that these things will one day get smarter than us, but we never had to deal with that, and if you want to know what life's like when you're not the apex intelligence, ask a chicken. (chicken clucks) So there's risks that come from people misusing AI, and then there's risks from AI getting super smart and deciding it doesn't need us.

   15. SBSteven Bartlett1:01

       Is that a real risk?

   16. GHGeoffrey Hinton1:02

       Yes, it is. But they're not gonna stop it 'cause it's too good for too many things.

   17. SBSteven Bartlett1:05

       What about regulations?

   18. GHGeoffrey Hinton1:06

       They have some, but they're not designed to deal with most of the threats, like the European regulations have a clause that say none of these apply to military uses of AI.

   19. SBSteven Bartlett1:14

       Really?

   20. GHGeoffrey Hinton1:14

       Yeah. It's crazy.

   21. SBSteven Bartlett1:16

       One of your students left OpenAI.

   22. GHGeoffrey Hinton1:18

       Yeah. He was probably the most important person behind the development of the early versions of ChatGPT, and I think he left 'cause he had safety concerns. We should recognize that this stuff is an existential threat, and we have to face the possibility that unless we do something soon, we're near the end.

   23. SBSteven Bartlett1:34

       So let's do the risks and what we end up doing in such a world. This has always blown my mind a little bit, 53% of you that listen to this show regularly haven't yet subscribed to the show, so could I ask you for a favor before we start? If you like the show and you like what we do here and you wanna support us, the free, simple way that you can do just that is by hitting the subscribe button. And my commitment to you is if you do that, then I'll do everything in my power, me and my team, to make sure that this show is better for you every single week. We'll listen to your feedback, we'll find the guests that you want me to speak to, and we'll continue to do what we do. Thank you so much. (instrumental music)

2. 2:11 – 4:20

   Why Do They Call You the Godfather of AI?

   1. SBSteven Bartlett2:11

      Geoffrey Hinton, they call you the godfather of AI.

   2. GHGeoffrey Hinton2:16

      Uh, yes, they do.

   3. SBSteven Bartlett2:17

      Why do they call you that?

   4. GHGeoffrey Hinton2:19

      There weren't that many people who believed that we could make neural networks work, artificial neural networks. So for a long time in AI, from the 1950s onwards, there were kind of two ideas about how to do AI. One idea was that sort of core of human intelligence was reasoning, and to do reasoning, you needed to use some form of logic, and so AI had to be based around logic, and in your head you must have something like symbolic expressions that you manipulated with rules, and that's how intelligence worked. And things like learning or reasoning by analogy, that'll come later once we've figured out how basic reasoning works. There was a different approach which is to say, let's model AI on the brain, 'cause obviously the brain makes us intelligent, so simulate a network of brain cells on a computer and try and figure out how you would learn strengths of connections between brain cells so that it learned to do complicated things, like recognize objects in images or recognize speech or even do reasoning. I pushed that approach for like 50 years. Because so few people believed in it, there weren't many good universities that had groups that did that, so if you did that, the best young students who believed in that came and worked with you. So I was very fortunate in getting a whole lot of really good students.

   5. SBSteven Bartlett3:44

      Some of which have gone on to create and play an instrumental role in creating platforms like OpenAI.

   6. GHGeoffrey Hinton3:49

      Yes. So Ilya Sutskever would be a nice example. A whole bunch of them.

   7. SBSteven Bartlett3:54

      Why did you believe that modeling it off the brain was a more effective approach?

   8. GHGeoffrey Hinton3:59

      It wasn't just me believed it. Early on, von Neumann believed it, and Turing believed it. And if either of those had lived, I think AI would have had a very different history. But they both died young.

   9. SBSteven Bartlett4:12

      You think AI would have been here sooner?

   10. GHGeoffrey Hinton4:14

       I think neural net, the neural net approach would have been accepted much sooner if either of them had lived.

3. 4:20 – 7:06

   Warning About the Dangers of AI

   1. GHGeoffrey Hinton4:20

   2. SBSteven Bartlett4:20

      In this season of your life, what mission are you on?

   3. GHGeoffrey Hinton4:25

      My main mission now is to warn people how dangerous AI could be.

   4. SBSteven Bartlett4:30

      Did you know that when you became the godfather of AI? (laughs)

   5. GHGeoffrey Hinton4:34

      No, not really. I was quite slow to understand some of the risks. Some of the risks were always very obvious, like people would use AI to make autonomous lethal weapons, that is things that go around deciding by themselves who to kill. Other risks like the idea that they will one day get smarter than us and maybe we'd become irrelevant, I was slow to recognize that. Other people recognized it 20 years ago. I only recognized it a few years ago that that was a real risk that was com- might be coming quite soon.

   6. SBSteven Bartlett5:05

      How could you not have foreseen that if, if with everything you know here about cracking the ability for these computers to learn similar to how humans learn, and just, you know, introducing any rate of improvement?

   7. GHGeoffrey Hinton5:18

      It's a very good question. How could you not have seen that? But remember neural networks 20, 30 years ago were very primitive in what they could do. They were nowhere near as good as humans at things like vision and language and speech recognition. The idea that you have to now worry about it getting smarter than people, that seems silly then.

   8. SBSteven Bartlett5:38

      When did that change?

   9. GHGeoffrey Hinton5:39

      It changed for the general population when ChatGPT came out. It changed for me when I realized that the kinds of digital intelligences we're making have something that makes them far superior to the kind of biological intelligence we have. If I want to share information with you... So I go off and I learn something, and I'd like to tell you what I learned.So I produce some sentences. This is a rather simplistic model, but roughly right. Your brain is trying to figure out, "How can I change the strengths of connections between neurons so I might to put that word next?" And so you'll do a lot of learning when a very surprising word comes, and not much learning when if it's a, when it's a very obvious word. If I say fish and chips, you don't do much learning when I say chips. But if I say fish and cucumber, you do a lot more learning. You wonder, "Why did I say cucumber?" So that's roughly what's going on in your brain.

   10. SBSteven Bartlett6:29

       I'm predicting what's coming next.

   11. GHGeoffrey Hinton6:31

       That's how we think it's working. Nobody really knows for sure how the brain works, and nobody knows how it gets the information about whether you should increase the strength of a connection or decrease the strength of a connection. That's the crucial thing. But what we do know now from AI is that if you could get information about whether to increase or decrease the connection strength so as to do better at whatever task you're trying to do, then we could learn incredible things, 'cause that's what we're doing now with artificial neural nets. It's just we don't know for real brains how they get that signal about whether to increase or decrease.

4. 7:06 – 10:33

   Concerns We Should Have About AI

   1. SBSteven Bartlett7:07

      As we sit here today, what are the big concerns you have around safety of AI? If we were to, to list the, the top couple that are really front of mind and that we should be thinking about, um ...

   2. GHGeoffrey Hinton7:17

      Can I have more than a couple?

   3. SBSteven Bartlett7:18

      Go ahead. I'll write them all down and we'll go through them.

   4. GHGeoffrey Hinton7:21

      Okay. First of all, I wanna make a distinction between two completely different kinds of risk. There's risks that come from people misusing AI.

   5. SBSteven Bartlett7:31

      Yeah.

   6. GHGeoffrey Hinton7:32

      And that's most of the risks and all of the short-term risks. And then there's risks that come from AI getting super smart and deciding it doesn't need us.

   7. SBSteven Bartlett7:41

      Is that a real risk?

   8. GHGeoffrey Hinton7:43

      And I talk mainly about that second risk because lots of people say, "Is that a real risk?"

   9. SBSteven Bartlett7:48

      Mm-hmm.

   10. GHGeoffrey Hinton7:48

       And yes, it is. Now, we don't know how much of a risk it is. We've never been in that situation before. We've never had to deal with things smarter than us. So really, the thing about that existential threat is that we have no idea how to deal with it, we have no idea what it's gonna look like, and anybody who tells you they know just what's gonna happen and how to deal with it, they're talking nonsense. So we don't know how to estimate the probabili- probabilities it'll replace us. Um, some people say it's, like, less than 1%. My friend Yann LeCun, who was a post-doc with me, thinks, "No, no, no, no. We're always gonna be... We build these things, we're always gonna be in control. We'll build them to be obedient." And other people, like Yudkowsky, say, "No, no, no. These things are gonna wipe us out for sure. If anybody builds it, it's gonna wipe us all out." And he's confident of that. I think both of those positions are extreme. It's very hard to estimate the probabilities in between.

   11. SBSteven Bartlett8:45

       If you had to bet on who was right out of your two friends?

   12. GHGeoffrey Hinton8:51

       I simply don't know. So if I had to bet, I'd say the probability's in between, and I don't know where to estimate in between. I often say 10 to 20% chance they'll wipe us out. But that's just gut, based on the idea that we're, we're still making them, and we're pretty ingenious. And the hope is that if enough smart people do enough research with enough resources, we'll figure out a way to build them so they'll never want to harm us.

   13. SBSteven Bartlett9:19

       Sometimes I think if we, we talk about that second, um, path, sometimes I think about nuclear bombs and the, the invention of the atomic bomb and how it compares. Like, how is this different? Because the atomic bomb came along, and I imagine a lot of people at, at that time thought, "Our days are numbered." Uh...

   14. GHGeoffrey Hinton9:32

       Uh, yes, I was there. We did. (laughs)

   15. SBSteven Bartlett9:35

       Yeah. But, but, but what's, what, h- we're still here.

   16. GHGeoffrey Hinton9:38

       We're still here, yes. So, the atomic bomb was really only good for one thing, and it was very obvious how it worked. Even if you hadn't had the pictures of Hiroshima and Nagasaki, it was obvious that it was a very big bomb that was very dangerous. With AI, it's good for many, many things. It's gonna be magnificent in healthcare and education, and more or less any industry that needs to use its data is gonna be able to use it better with AI. So we're not gonna stop the development. You know, people say, "Well, why don't we just stop it now?" We're not gonna stop it 'cause it's too good for too many things. Also, we're not gonna stop it 'cause it's good for battle robots, and none of the countries that sell weapons are gonna want to stop it. Like the European regulations, they have some regulations about AI, and it's good they have some regulations.

5. 10:33 – 12:12

   European AI Regulations

   1. GHGeoffrey Hinton10:33

      But they're not designed to deal with most of the threats. And in particular, the European regulations have a, a clause in them that say none of these regulations apply to military uses of AI. So governments are willing to regulate, regulate companies and people, but they're not willing to regulate themselves.

   2. SBSteven Bartlett10:53

      It seems pretty crazy to me that they, uh... I, I go back and forward, but if Europe has a regulation, but the rest of the world doesn't, aren't we just putting ourselves-

   3. GHGeoffrey Hinton11:01

      Yeah, puts them at a competitive disadvantage.

   4. SBSteven Bartlett11:02

      Yeah.

   5. GHGeoffrey Hinton11:03

      So-

   6. SBSteven Bartlett11:03

      And we're seeing this already. I don't think people realize that when OpenAI release a new model or a new piece of software in America, they can't release it to the, to Europe yet because of regulations here. So Sam Altman tweeted saying, "Our new AI agent thing is available to everybody, but it can't come to Europe yet because there's regulations."

   7. GHGeoffrey Hinton11:20

      Yes.

   8. SBSteven Bartlett11:20

      And what does that do? That gives us a productive disadvantage? Productivity disadvantage?

   9. GHGeoffrey Hinton11:24

      Yes.

   10. SBSteven Bartlett11:24

       Right.

   11. GHGeoffrey Hinton11:25

       What we need is... I mean, at this point in history, when we're about to produce things more intelligent than ourselves, what we really need is a kinda world government that works run by intelligent, thoughtful people. And that's not what we've got.

   12. SBSteven Bartlett11:40

       So, a free-for-all?

   13. GHGeoffrey Hinton11:42

       Well, that, what we've got is sort of, we've got capitalism, which has done very nicely by us. It's produced lots of goods, goods and services for us. But these big companies, they're legally required to try and maximize profits.

   14. SBSteven Bartlett12:00

       Mm-hmm.

   15. GHGeoffrey Hinton12:00

       And that's not what you want from the people developing this stuff.

   16. SBSteven Bartlett12:05

       So let's do the risks, then. You talked about there's human risks and then there's lung-

   17. GHGeoffrey Hinton12:08

       So, I've distinguished these two kinds of risk.

   18. SBSteven Bartlett12:10

       Yeah.

   19. GHGeoffrey Hinton12:10

       Let's talk about all the risks from

6. 12:12 – 14:25

   Cyber Attack Risk

   1. GHGeoffrey Hinton12:12

      bad human actors using AI. There's cyber attacks. So between 2023 and 2024, they increased by about a factor of 12, 1,200%. And that's probably because these large language models make it much easier to do phishing attacks.

   2. SBSteven Bartlett12:33

      And a ph- a phishing attack, for anyone that doesn't know, is?

   3. GHGeoffrey Hinton12:36

      It's, they send you something saying, uh, "Hi, I'm your friend John and I'm stuck in El Salvador. Could you just wire this money?" That's one kind of attack.

   4. SBSteven Bartlett12:45

      Yeah.

   5. GHGeoffrey Hinton12:45

      But the phishing attacks are really trying to get your logon credentials.

   6. SBSteven Bartlett12:49

      And now with AI, they can clone my voice, my image.

   7. GHGeoffrey Hinton12:51

      They can do all that.

   8. SBSteven Bartlett12:52

      I'm struggling at the moment because there's a bunch of AI scams on X and also Meta, and there's one in particular on Meta, so Instagram, Facebook at the moment, which is a paid advert where they've taken my voice from the podcast, they've taken the, my mannerisms and they've made a new video of me encouraging people to go and take part in this crypto Ponzi scam or whatever. And we've been, you know, we spent weeks and weeks and weeks and weeks on end emailing Meta, telling, "Please take this down." They take it down, another one pops up. They take that one down, another one pops up. So it's like whack-a-mole.

   9. GHGeoffrey Hinton13:21

      Yes. Very annoying.

   10. SBSteven Bartlett13:22

       The, the heartbreaking part is you get the messages from people that have fallen for the scam-

   11. GHGeoffrey Hinton13:25

       Right.

   12. SBSteven Bartlett13:25

       ... and they've lost 500 pounds or $500 and-

   13. GHGeoffrey Hinton13:27

       And they're cross with you 'cause you recommended it. (laughs)

   14. SBSteven Bartlett13:29

       And I'm, I'm like, I'm sad for them.

   15. GHGeoffrey Hinton13:31

       It's very annoying.

   16. SBSteven Bartlett13:32

       Yeah.

   17. GHGeoffrey Hinton13:32

       I have a, a smaller version of that, which is p- some people now publish papers with me as one of the authors.

   18. SBSteven Bartlett13:39

       Mm-hmm.

   19. GHGeoffrey Hinton13:40

       And it looks like it's in order that they can get lots of citations to themselves.

   20. SBSteven Bartlett13:44

       Ah. So cyber attacks, a very real threat. There's been an explosion of those.

   21. GHGeoffrey Hinton13:49

       Yes. And these already, obviously AI is very patient, so they can go through 100 million lines of code looking for known ways of attacking them. That's easy to do. But they're gonna get more creative and they may, some people believe, and I ... Some people who know a lot believe that maybe by 2030, they'll be creating new kinds of cyber attacks, which no person ever thought of. So that's very worrisome.

   22. SBSteven Bartlett14:18

       Because they can think for themselves and discover new ways to attack.

   23. GHGeoffrey Hinton14:19

       They can think for themselves. They can draw new conclusions from much more data than a person ever

7. 14:25 – 16:12

   How to Protect Yourself From Cyber Attacks

   1. GHGeoffrey Hinton14:25

      saw.

   2. SBSteven Bartlett14:26

      Is there anything you are doing to protect yourself from cyber attacks at all?

   3. GHGeoffrey Hinton14:30

      Yes. It's one of the few places where I changed what I do radically because I'm scared of cyber attacks. Canadian banks are extremely safe. In 2008, no Canadian banks came anywhere near going bust.

   4. SBSteven Bartlett14:44

      Mm-hmm.

   5. GHGeoffrey Hinton14:45

      So they're very safe banks 'cause they're well regulated, fairly well regulated. Nevertheless, I think a cyber attack might be able to bring down a bank. Now, if you have, all my savings are in shares in banks, s- held by banks. So if the bank gets attacked and it holds your shares, they're still your shares. And so I think you'd be okay unless the attacker sells the shares, 'cause the bank can sell the shares. If the attacker sells your shares, I think you're screwed. I don't know. I mean, maybe the bank would have to try and reimburse you, but the bank's bust by now, right? So...

   6. SBSteven Bartlett15:23

      Mm-hmm.

   7. GHGeoffrey Hinton15:24

      So I'm worried about a Canadian bank being taken down by a cyber attack and the attacker selling, selling shares that it holds. So I spread my money and my children's money between three banks in the belief that if a cyber attack takes down one Canadian bank, the other Canadian banks will very quickly get very careful.

   8. SBSteven Bartlett15:46

      And do you have a phone that's not connected to the internet? Do you have any, like, you know, I'm thinking about storing data and stuff like that. Do you think it's wise to consider having cold storage?

   9. GHGeoffrey Hinton15:55

      I have a little disc drive and I back up my laptop on this hard drive.

   10. SBSteven Bartlett16:00

       Mm-hmm.

   11. GHGeoffrey Hinton16:00

       So I actually have everything on my laptop on a hard drive. At least, you know, if the whole internet went down, I had the sense I still got it on my laptop and I still got my information.

   12. SBSteven Bartlett16:11

       Yeah.

8. 16:12 – 17:26

   Using AI to Create Viruses

   1. SBSteven Bartlett16:12

   2. GHGeoffrey Hinton16:12

      Then the next thing is using AIs to create nasty viruses.

   3. SBSteven Bartlett16:18

      Okay.

   4. GHGeoffrey Hinton16:19

      And the problem with that is that req- just requires one crazy guy with a grudge, one guy who knows a little bit of molecular biology, knows a lot about AI and just wants to destroy the world. You can now create new viruses relatively cheaply using AI. And you don't have to be a very skilled molecular biologist to do it. And that's very scary. So you could have a small cult, for example. A, a small cult might be able to raise a few million dollars. For a few million dollars, they might be able to design a whole bunch of viruses.

   5. SBSteven Bartlett16:54

      Well, I'm thinking about some of our foreign adversaries doing government funded programs. I mean, there was lots of talk around COVID and Wu, the Wuhan Laboratory and what they were doing in gain-of-function research. But I'm wondering if in, you know, a China or a Russia or an Iran or something, the government could fund a program for a small group of scientists to make a virus that they could, you know?

   6. GHGeoffrey Hinton17:14

      I think they could. Yes. Now, they'd be worried about retaliation. They'd be worried about other governments doing the same to them. Hopefully that would help keep it under control. They might also be worried about the virus spreading to their country.

   7. SBSteven Bartlett17:25

      Okay.

9. 17:26 – 19:03

   AI and Corrupt Elections

   1. SBSteven Bartlett17:26

   2. GHGeoffrey Hinton17:26

      Then there's, um, corrupting elections.

   3. SBSteven Bartlett17:30

      Okay.

   4. GHGeoffrey Hinton17:31

      So if you wanted to use AI to corrupt elections, a very effective thing is to be able to do targeted political advertisements where you know a lot about the person. So, anybody who wanted to use AI for corrupting elections would try and get as much data as they could about everybody in the electorate. With that in mind, it's a bit worrying what Musk is doing at present in the States, going in and insisting on getting access to all these things that were very carefully siloed. The claim is it's to make things more efficient, but it's exactly what you would want if you intended to corrupt the next election.

   5. SBSteven Bartlett18:10

      How do you mean? 'Cause you get all this data on people-

   6. GHGeoffrey Hinton18:12

      You get all this data on people-

   7. SBSteven Bartlett18:14

      Yeah.

   8. GHGeoffrey Hinton18:14

      ... you know how much they make, where they l- You know everything about them. Once you know that, it's very easy to manipulate them.

   9. SBSteven Bartlett18:20

      Because you can make an AI that-

   10. GHGeoffrey Hinton18:22

       You can send messages, um, that they'll find very convincing, telling them not to vote, for example. So I have no, no reason other than common sense to think this, but I wouldn't be surprised if part of the motivation of getting all this data from American government sources is to corrupt elections. Another part might be that it's very nice training data for a big model.

   11. SBSteven Bartlett18:48

       But he would have to be taking that data from the government and feeding it into his-

   12. GHGeoffrey Hinton18:52

       Yes. And what they've done is turned off lots of the security controls, got rid of the, some of the organization to protect against that.

   13. SBSteven Bartlett19:01

       Um, so that's corrupting elections.

10. 19:03 – 22:48

    How AI Creates Echo Chambers

    1. SBSteven Bartlett19:03

    2. GHGeoffrey Hinton19:03

       Okay. Then there's creating these two echo chambers by organizations like YouTube and Facebook, showing people things that will make them indignant. People love to be indignant.

    3. SBSteven Bartlett19:20

       Indignant as in angry? Or what does indignant mean?

    4. GHGeoffrey Hinton19:23

       Feeling, um, sort of angry, but feeling righteous.

    5. SBSteven Bartlett19:27

       Okay.

    6. GHGeoffrey Hinton19:28

       So for example, if you were to show me something that said, "Trump did this crazy thing. Here's a video of Trump doing this completely crazy thing." I would immediately click on it. (laughs)

    7. SBSteven Bartlett19:38

       Yeah. (laughs) Okay, so putting us in echo chambers and dividing us.

    8. GHGeoffrey Hinton19:43

       Yes. And that's, um, the policy that YouTube and Facebook and others use for deciding what to show you next is causing that. If they had a policy of showing you balanced things, they wouldn't get so many clicks and they wouldn't be able to sell so many advertisements.

    9. SBSteven Bartlett20:02

       Definitely.

    10. GHGeoffrey Hinton20:02

        And so it's basically the profit motive is saying, "Show 'em whatever'll make 'em click." And what'll make 'em click is things that are more and more extreme.

    11. SBSteven Bartlett20:12

        And that confirm my existing bias.

    12. GHGeoffrey Hinton20:14

        That confirm my existing bias. So you're getting your biases confirmed all the time.

    13. SBSteven Bartlett20:18

        Further and further and further and further.

    14. GHGeoffrey Hinton20:19

        Yeah, yeah.

    15. SBSteven Bartlett20:19

        Which means you're d- you're driving away from people.

    16. GHGeoffrey Hinton20:21

        Which is now there's, in the States there's two communities that don't hardly talk to each other.

    17. SBSteven Bartlett20:25

        I'm not sure people realize that this is actually happening every time they open an app, but if you go on a TikTok or a YouTube or, or one of these big social networks, the algorithm, as you, you said, is designed to show you more of the things that you had interest in last time. So if you just play that out over 10 years, it's gonna drive you further and further and further into whatever ideology or belief you have, and further away from nuance and common sense and, um, parity. Which is a pretty remarkable thing. I, I, like people don't know it's happening. They just open their phones and experience something and think this is the news or the experience everyone else is having.

    18. GHGeoffrey Hinton21:00

        Right. So basically if you have a newspaper and everybody gets the same newspaper-

    19. SBSteven Bartlett21:05

        Yeah.

    20. GHGeoffrey Hinton21:06

        ... you get to see all sorts of things you weren't looking for, and you get a sense that if it's in the newspaper, it's an important thing or significant thing. But if you have your own news feed, my news feed on my iPhone, three-quarters of the stories are about AI.

    21. SBSteven Bartlett21:20

        (laughs)

    22. GHGeoffrey Hinton21:20

        And I find it very hard to know if the whole world's talking about AI all the time or if it's just my news feed.

    23. SBSteven Bartlett21:26

        (laughs) Okay, so driving me into my echo chambers, um, which is gonna continue to divide us further and further. I'm actually noticing that the algorithms are becoming even more, what's the word? Tailored. And people might go, "That's great." But what it means is they're becoming even more personalized, which means, it means that my reality is becoming even further from your reality.

    24. GHGeoffrey Hinton21:49

        Yeah, yeah. It's crazy. We don't have a shared reality anymore. I share reality with other people who watch the BBC and other, BBC News and other people who read The Guardian and other people who read The New York Times. I have almost no shared reality with people who watch Fox News.

    25. SBSteven Bartlett22:08

        It's pretty, it's pretty, um, I, I, I-

    26. GHGeoffrey Hinton22:10

        It's worrisome.

    27. SBSteven Bartlett22:11

        Yeah.

    28. GHGeoffrey Hinton22:12

        Behind all this is the idea that these companies just wanna make profit, and they'll do whatever it takes to make more profit.

    29. SBSteven Bartlett22:18

        Because they have to.

    30. GHGeoffrey Hinton22:20

        They're legally obliged to do that.
11. 22:48 – 24:31

    Regulating New Technologies

    1. GHGeoffrey Hinton22:48

       not things that are bad for people in general. So once you get to a situation where in order to make more profit, the company starts doing things that are very bad for society, like showing you things that are more and more extreme, that's what regulations are for. So you need regulations with capitalism. Now companies will always say, "Regulations get in the way, make us less efficient." And that's true. The whole point of regulations is to stop them doing things to make profit that hurts society. And we need strong regulation.

    2. SBSteven Bartlett23:17

       Who's gonna decide whether it hurts society or not? Because, you know-

    3. GHGeoffrey Hinton23:21

       That's the job of politicians. Unfortunately, if the politicians are owned by the companies, that's not so good.

    4. SBSteven Bartlett23:27

       And also the politicians might not understand the technology. We've pro- you've probably seen the Senate hearings where they wheel out, you know, Mark Zuckerberg and these big tech CEOs. And it is quite embarrassing because they're asking their own questions.

    5. GHGeoffrey Hinton23:37

       Well, I've seen the video of the US Education Secretary talking about how they're gonna get AI in the classrooms, except she thought it was called A1.

    6. SBSteven Bartlett23:48

       Ah.

    7. GHGeoffrey Hinton23:48

       She's actually there saying, "We're gonna have all the kids interacting with A1."

    8. NANarrator23:53

       There is a school system that's gonna start, um, making sure that first graders or even pre-Ks have A1 teaching, you know, every year starting, you know, that far down in the grades. And that's just a, that's a wonderful thing.

    9. SBSteven Bartlett24:07

       (laughs)

    10. GHGeoffrey Hinton24:07

        (laughs)

    11. SBSteven Bartlett24:10

        And these are, what, these are the people that...

    12. GHGeoffrey Hinton24:12

        These are the people in charge.

    13. SBSteven Bartlett24:14

        ... ultimately the tech companies are in charge because they were outsmart-

    14. GHGeoffrey Hinton24:17

        Well, the tech companies in the States now, at least a few weeks ago when I was there, they were running an advertisement about how it was very important not to regulate AI because it would hurt us in the competition with China.

12. 24:31 – 25:57

    Are Regulations Holding Us Back From Competing With China?

    1. GHGeoffrey Hinton24:31

    2. SBSteven Bartlett24:31

       Yeah. And that's a, that's a plausible argument, no?

    3. GHGeoffrey Hinton24:34

       Yes, it will. But you have to decide, do you want to compete with China by doing things that will do a lot of harm to your society? And you probably don't.

    4. SBSteven Bartlett24:49

       I guess they would say that it's not just China, it's Denmark and Australia and Canada and the UK.

    5. GHGeoffrey Hinton24:55

       Yeah. They're, they're not so worried about...

    6. SBSteven Bartlett24:57

       And Germany. But if they kneecap themselves with regulation, if they slowed themselves down, then the founders, the entrepreneurs, the investors are gonna go invest-

    7. GHGeoffrey Hinton25:03

       I think calling it kneecapping is taking-

    8. SBSteven Bartlett25:05

       Yeah.

    9. GHGeoffrey Hinton25:05

       ... a particular point of view, is take- taking the point of view that regulations are sort of very harmful. What you need to do is just constrain the big companies so that in order to make profit, they have to do things that are socially useful. Like Google Search is a great example. That didn't need regulation because it just made information available to people. It was great. But then if you take YouTube, which starts showing you adverts and showing you more and more extreme things, that needs regulation.

    10. SBSteven Bartlett25:35

        But we don't have the people to regulate it, as we've identified.

    11. GHGeoffrey Hinton25:38

        I think people know pretty well, um, that particular problem of showing you more and more extreme things, that's a well-known problem that the politicians understand. They just, um, need to get on and regulate it.

    12. SBSteven Bartlett25:50

        So that was the, the next point, which was that the algorithms are going to drive us further into our echo chambers.

    13. GHGeoffrey Hinton25:55

        Right.

    14. SBSteven Bartlett25:56

        What's next?

13. 25:57 – 28:33

    The Threat of Lethal Autonomous Weapons

    1. SBSteven Bartlett25:57

    2. GHGeoffrey Hinton25:57

       Lethal autonomous weapons.

    3. SBSteven Bartlett26:00

       Lethal autonomous weapons.

    4. GHGeoffrey Hinton26:03

       That means things that can kill you and make their own decision about whether to kill you.

    5. SBSteven Bartlett26:08

       Which is the great dream, I guess, of the military industrial complex, being able to create-

    6. GHGeoffrey Hinton26:13

       Yes.

    7. SBSteven Bartlett26:14

       ... such weapons.

    8. GHGeoffrey Hinton26:14

       So the worst thing about them is big powerful countries always have the abil- ability to invade smaller, poorer countries. They're just more powerful.

    9. SBSteven Bartlett26:25

       Mm-hmm.

    10. GHGeoffrey Hinton26:26

        But if you do that using actual soldiers, you get bodies coming back in bags, and the relatives of the soldiers who were killed don't like it. So you get something like Vietnam.

    11. SBSteven Bartlett26:38

        Mm-hmm.

    12. GHGeoffrey Hinton26:39

        In the end, there's a lot of protests at home. If instead of bodies coming back in bags, it was dead robots, there'd be much less protest and the military industrial complex would like it much more, because robots are expensive. And suppose you had something that could get killed and was expensive to replace, that would be just great. Big countries can invade small countries much more easily because they don't have their soldiers being killed.

    13. SBSteven Bartlett27:07

        And the risk here is that these robots will malfunction or they'll just be more and more-

    14. GHGeoffrey Hinton27:14

        No, no. That's ... Even if the robots do exactly what the people who built the robots want them to do, the risk is that it's going to make big countries invade small countries more often.

    15. SBSteven Bartlett27:22

        More often because they can.

    16. GHGeoffrey Hinton27:24

        Yeah. And it's not a nice thing to do.

    17. SBSteven Bartlett27:25

        So it brings down the friction of war.

    18. GHGeoffrey Hinton27:26

        It brings down the cost of doing an invasion.

    19. SBSteven Bartlett27:29

        Mm-hmm. But, and these machines will be smarter at warfare as well. So they'll be better-

    20. GHGeoffrey Hinton27:34

        Well, even when the machines aren't smarter. So the lethal autonomous weapons, they can make them now, and they ... I think all the big defense departments are busy making them. Even if they're not smarter than people, they're still very nasty, scary things.

    21. SBSteven Bartlett27:48

        Because I'm thinking that, you know, they could show just a picture, "Go get this guy."

    22. GHGeoffrey Hinton27:53

        Yeah.

    23. SBSteven Bartlett27:53

        And go take out anyone he's been texting. And this little wasp...

    24. GHGeoffrey Hinton27:58

        So two days ago, I was visiting a friend of mine in Sussex who had a drone that cost less than 200 pounds. And the drone went up, it took a good look at me, and then it could follow me through the woods.

    25. SBSteven Bartlett28:13

        Mm-hmm.

    26. GHGeoffrey Hinton28:13

        And it followed ... It was very spooky having this drone, it was about two meters behind me, it was looking at me, and if I moved over there, it moved over there, it could just track me-

    27. SBSteven Bartlett28:21

        Mm-hmm.

    28. GHGeoffrey Hinton28:22

        ... for 200 pounds. But it was already quite spooky.

    29. SBSteven Bartlett28:26

        Yeah, and I imagine there's, as you say, a race going on as we speak to-

    30. GHGeoffrey Hinton28:29

        Yeah.
14. 28:33 – 30:15

    Can These AI Threats Combine?

    1. SBSteven Bartlett28:33

       There is a, a risk I often hear that some of these things will combine and the cyber attack will release weapons.

    2. GHGeoffrey Hinton28:41

       Sure. Um, you can, you can get combinatorially many risks by combining these other risks.

    3. SBSteven Bartlett28:47

       Mm-hmm.

    4. GHGeoffrey Hinton28:48

       So, I mean, for example, you could get a super intelligent AI that decides to get rid of people. And the obvious way to do that is just to make one of these nasty viruses. If you made a virus that was very contagious, very lethal, and very slow, everybody would have it before they realized what was happening. I mean, I think if a super intelligence wanted to get rid of us, it will probably go for something biological like that, that wouldn't affect it.

    5. SBSteven Bartlett29:15

       Do you not think it could just very quickly turn us against each other? For example, it could send a warning on the nuclear systems in America that there's a nuclear bomb coming from Russia or vice versa, and one retaliates.

    6. GHGeoffrey Hinton29:26

       Yeah. I mean, my basic view is there's so many ways in which a super intelligence could get rid of us. It's not worth speculating about.

    7. SBSteven Bartlett29:35

       What, what is to stop it?

    8. GHGeoffrey Hinton29:37

       What you have to do is prevent it ever wanting to. That's what we should be doing research on. There's no way we're going to prevent it from... It's smarter than us, right? There's no way we're going to prevent it getting rid of us if it wants to. We're not used to thinking about things smarter than us. If you want to know what life's like when you're not the apex intelligence, ask a chicken.

    9. SBSteven Bartlett30:03

       (laughs) Yeah. I was thinking of my dog Pablo, my French bulldog, this morning as I left home. He has no idea where I'm going.

    10. GHGeoffrey Hinton30:08

        Right.

    11. SBSteven Bartlett30:08

        He has no idea what I do.

    12. GHGeoffrey Hinton30:10

        Right.

    13. SBSteven Bartlett30:10

        I can't even talk to him.

    14. GHGeoffrey Hinton30:12

        Yeah. And the g- the intelligence gap will be like that.

15. 30:15 – 32:01

    Restricting AI From Taking Over

    1. SBSteven Bartlett30:15

       So you're telling me that if I'm Pablo, my French bulldog, I need to figure out a way to make my owner not wipe me out?

    2. GHGeoffrey Hinton30:24

       Yeah. So we have one example of that, which is mothers and babies. Evolution put a lot of work into that. Mothers are smarter than babies, but babies are in control and they're in control 'cause the mother just can't bear lots of hormones and things. But the bab- the mother just can't bear the sound of the baby crying.

    3. SBSteven Bartlett30:42

       Not all mothers.

    4. GHGeoffrey Hinton30:43

       Not all mothers. And then the baby's not in control and then bad things happen. We somehow need to figure out how to make them not want to take over. The analogy I often use is, forget about intelligence, just think about physical strength. Suppose you have a nice little tiger cub. It's sort of a bit bigger than a cat. It's really cute. It's very cuddly, very interesting to watch. Accept that you better be sure that when it grows up, it never wants to kill you 'cause it ever wanted to kill you, you'd be dead in a few seconds.

    5. SBSteven Bartlett31:15

       And you're saying the AI we have now is the tiger cub?

    6. GHGeoffrey Hinton31:17

       Yep.

    7. SBSteven Bartlett31:18

       And it's growing up.

    8. GHGeoffrey Hinton31:19

       Yep.

    9. SBSteven Bartlett31:21

       So we need to train it as it's, when it's a baby.

    10. GHGeoffrey Hinton31:23

        Well, no, a tiger has lots of innate stuff built in. So you know when it grows up, uh, it's not a safe thing to have around.

    11. SBSteven Bartlett31:29

        But lions, people that have lions as pets-

    12. GHGeoffrey Hinton31:32

        Yes.

    13. SBSteven Bartlett31:32

        ... sometimes the lion is affectionate to its creator, but not to others.

    14. GHGeoffrey Hinton31:35

        Yes. And we don't know whether these AIs ... We, we simply don't know whether we can make them not want to take over and not want to hurt us.

    15. SBSteven Bartlett31:45

        Do you think we can? Do you think it's possible to train superintelligence?

    16. GHGeoffrey Hinton31:48

        I don't know. I don't think it's clear that we can. So I think it might be hopeless, but I also think we might be able to, and it'd be sort of crazy if people went extinct 'cause we couldn't be bothered to try.

16. 32:01 – 33:45

    Reflecting on Your Life’s Work Amid AI Risks

    1. GHGeoffrey Hinton32:01

    2. SBSteven Bartlett32:01

       If that's even a possibility, how do you feel about your life's work? Because you were ...

    3. GHGeoffrey Hinton32:06

       Yeah. Um, it sort of takes the edge off it, doesn't it? (laughs)

    4. SBSteven Bartlett32:11

       Yeah. (laughs)

    5. GHGeoffrey Hinton32:11

       I mean, the idea is gonna be wonderful in healthcare and wonderful in education and wonderful ... I mean, it's gonna make call centers much more efficient, though one worries a bit about what the people who are doing that job now do. It makes me sad. I don't feel particularly guilty about developing AI, like, 40 years ago because at that time, we had no idea that this stuff was gonna happen this fast. We thought we had plenty of time to worry about things like that. They ... When you, when you can't get the AI to do much and you want to get it to do a little bit more, you don't worry about this stupid little thing is gonna take over from people. You just want it to be able to do a little bit more of the things people can do. It's not like I knowingly did something thinking, "This might wipe us all out, but I'm gonna do it anyway."

    6. SBSteven Bartlett32:56

       Mm-hmm.

    7. GHGeoffrey Hinton32:58

       But it is a bit sad that it's not just gonna be something for good. So I feel I have a duty now to talk about the risks.

    8. SBSteven Bartlett33:07

       And if you could play it forward and you could go forward 30, 50 years, and you found out that it led to the extinction of humanity, and if that does end up being, (laughs) being the outcome?

    9. GHGeoffrey Hinton33:21

       Well, if you played it forward and it led to the extinction of humanity, I would use that to tell people to tell their governments that we really have to work on how we are gonna keep this stuff under control. I think we need people to tell governments that governments have to force the companies to use their resources to work on safety. And they're not doing much of that because you don't make profits that way.

17. 33:45 – 37:49

    Student Leaving OpenAI Over Safety Concerns

    1. GHGeoffrey Hinton33:45

    2. SBSteven Bartlett33:45

       One of your, your students we talked about earlier, um, I- Ilya?

    3. GHGeoffrey Hinton33:49

       Yep.

    4. SBSteven Bartlett33:50

       Ilya left OpenAI.

    5. GHGeoffrey Hinton33:52

       Yep.

    6. SBSteven Bartlett33:53

       And there was lots of conversation around the fact that he left because he had safety concerns.

    7. GHGeoffrey Hinton33:57

       Yes.

    8. SBSteven Bartlett33:58

       And he's gone on to set, set up a AI safety company.

    9. GHGeoffrey Hinton34:02

       Yes.

    10. SBSteven Bartlett34:04

        Why do you think he left?

    11. GHGeoffrey Hinton34:06

        I think he left 'cause he had safety concerns.

    12. SBSteven Bartlett34:08

        Really?

    13. GHGeoffrey Hinton34:08

        He ... Um, I still have lunch with him from time to time.

    14. SBSteven Bartlett34:12

        Oh, okay.

    15. GHGeoffrey Hinton34:12

        His, his parents live in Toronto. When he comes to Toronto, we have lunch together. He doesn't talk to me about what went on at OpenAI, so I have no inside information about that. But I know Ilya very well and he is genuinely concerned with safety. So I think that's why he left.

    16. SBSteven Bartlett34:26

        Because he was one of the top people. I mean, he was-

    17. GHGeoffrey Hinton34:28

        He was probably the most important person behind the development of, um, ChatGPT, the, the early versions like GPT-2. He was very important in the development of that.

    18. SBSteven Bartlett34:38

        You know him personally, so you know his character.

    19. GHGeoffrey Hinton34:41

        Yes. He has a good moral compass. He's not like someone like Musk, who has no moral compass.

    20. SBSteven Bartlett34:47

        Does Sam Altman have a good moral compass?

    21. GHGeoffrey Hinton34:50

        We'll see. (laughs)

    22. SBSteven Bartlett34:53

        Y-

    23. GHGeoffrey Hinton34:53

        I don't know Sam, so I don't want to comment on that.

    24. SBSteven Bartlett34:57

        But from what you've seen, are you concerned about the actions that they've taken?

    25. GHGeoffrey Hinton35:02

        Well, because-

    26. SBSteven Bartlett35:02

        'Cause if you know Ilya, and Ilya's a good guy and he's left ... (laughs)

    27. GHGeoffrey Hinton35:06

        ... that would give you some insight, yes. It would give you some reason to believe that there's a problem there. And if you look at Sam's statements some years ago, he sort of happily said in one interview, "And this stuff will probably kill us all." That's not exactly what he said, but that's what it amounted to. Now he's saying, "You don't need to worry too much about it." And I suspect that's not driven by seeking after the truth. That's driven by seeking after money.

    28. SBSteven Bartlett35:37

        Is it money or is it power?

    29. GHGeoffrey Hinton35:39

        Yeah. I shouldn't have said money. It's some, some combination of those, yes.

    30. SBSteven Bartlett35:43

        Okay, 'cause money's a proxy for power, but ... I am ... I've got a friend who's a billionaire, and he is in those circles. And when I went to his house and had, uh, lunch with him one day, he knows lots of people in AI building the biggest AI companies in the world. And he gave me a cautionary warning across the, uh, across his kitchen table in London, where he gave me an insight into the private conversations these people have. Not the media interviews they do where they talk about safety and all these things, but actually what some of these individuals think is gonna happen.
18. 37:49 – 39:51

    Are You Hopeful About the Future of AI?

    1. SBSteven Bartlett37:49

       So coming back to this point of the possibility of destruction and the motives of these big companies, are you at all hopeful that anything can be done to slow down the pace and acceleration of AI?

    2. GHGeoffrey Hinton38:07

       Okay. There's two issues. One is, can you slow it down?

    3. SBSteven Bartlett38:10

       Yeah.

    4. GHGeoffrey Hinton38:11

       And the other is, can you make it so, uh, it will be safe in the end? It won't wipe us all out. I don't believe we're gonna slow it down.

    5. SBSteven Bartlett38:20

       Yeah.

    6. GHGeoffrey Hinton38:20

       And the reason I don't believe we're gonna slow it down is because there's competition between countries and competition between companies within a country, and all of that is making it go faster and faster. And if the US slowed it down, China wouldn't slow it down.

    7. SBSteven Bartlett38:34

       Does Ilya think it's possible to make AI safe?

    8. GHGeoffrey Hinton38:40

       I think he does. He won't tell me what his secret sauce is. I th- I'm not sure how many people know what his secret sauce is. I think a lot of the investors don't know what his secret sauce is, but they've given him billions of dollars anyway 'cause they have so much faith in Ilya, which isn't foolish. I mean, he was very important in AlexNet, which got object recognition working well. He was the main, the main force behind the things like GPT-2, which then led to ChatGPT. So I think having a lot of faith in Ilya is a very reasonable decision.

    9. SBSteven Bartlett39:14

       There's something quite haunting about the guy that made and was the main force behind GPT-2, which led rise to this whole revolution, left the company because of safety reasons. He knows something that I don't know (laughs) about what might happen next.

    10. GHGeoffrey Hinton39:29

        Well, the company had... Now, I don't know the precise details, um, but I'm fairly sure the company had indicated that would, it would use a significant fraction of its resources, of the compute time-

    11. SBSteven Bartlett39:42

        Mm-hmm.

    12. GHGeoffrey Hinton39:42

        ... for doing safety research, and then it kept, then it reduced that fraction. I think that's one of the things that happened.

    13. SBSteven Bartlett39:48

        Yeah. That was reported publicly.

    14. GHGeoffrey Hinton39:49

        Yes.

    15. SBSteven Bartlett39:49

        Yeah.

19. 39:51 – 42:47

    The Threat of AI-Induced Joblessness

    1. SBSteven Bartlett39:51

       We'd gotten to the autonomous weapons part of the RISK framework.

    2. GHGeoffrey Hinton39:56

       Right. So the next one is joblessness.

    3. SBSteven Bartlett39:59

       Yeah.

    4. GHGeoffrey Hinton40:00

       In the past, new technologies have come in which didn't lead to joblessness. New jobs were created. So the classic example people use is automatic teller machines. When automatic teller machines came in, a lot of bank tellers didn't lose their jobs. They just got to do more interesting things. But here, I think this is more like when they got machines in the Industrial Revolution, and you can't have a job digging ditches now because a machine can dig ditches much better than you can.

    5. SBSteven Bartlett40:31

       Mm-hmm.

    6. GHGeoffrey Hinton40:32

       And I think for mundane intellectual labor, AI is just going to replace everybody. Now, it will, may well be in the form of you have fewer people using AI assistants, so it's a combination of a person and an AI assistant are now doing the work that 10 people could do previously.

    7. SBSteven Bartlett40:52

       People say that it will create new jobs though, so we'll be fine.

    8. GHGeoffrey Hinton40:56

       Yes, and that's been the case for other technologies, but this is a very different kind of technology. If it can do all mundane human intellectual labor, then what new jobs is it gonna create? You'd ha- you'd have to be very skilled to have a job that it couldn't just do. So I don't th- I don't think they're right. I think you can try and generalize from other technologies that come in, like computers or automatic teller machines, but I think this is different.

    9. SBSteven Bartlett41:23

       People use this phrase. They say, "AI won't take your job. A human using AI will take your job."

    10. GHGeoffrey Hinton41:27

        Yes. I think that's true. But for s- many jobs, that'll mean you need far fewer people. My niece answers letters of complaint to a health service. It used to take her 25 minutes. She'd read the complaint, and she'd think how to reply, and she'd write a letter and... Now, she just scans it into, um, a chatbot, and it writes the letter. She just checks the letter. Occasionally, she tells it to revise it in some ways. The whole process takes her five minutes. That means she can answer five times as many letters, and that means they need five times fewer of her, so she can do the job that five of her used to do. Now, that will mean they need less people. In other jobs, like in healthcare, they're much more elastic. So if you could make doctors five times as efficient, we could all have five times as much healthcare for the same price, and that would be great. There's, there's almost no limit to how much healthcare people can absorb.

    11. SBSteven Bartlett42:30

        Mm-hmm.

    12. GHGeoffrey Hinton42:30

        They always want more healthcare-

    13. SBSteven Bartlett42:32

        Mm-hmm.

    14. GHGeoffrey Hinton42:32

        ... if, if there's no cost to it. There are jobs where you can make a person with an AI assistant much more efficient and you won't lead to...... less people, because you'll just have much more of that being done. But most jobs, I think, are not like that.

20. 42:47 – 44:38

    If Muscles and Intelligence Are Replaced, What’s Left?

    1. GHGeoffrey Hinton42:47

    2. SBSteven Bartlett42:47

       Am I right in thinking the sort of Industrial Revolution w- played a role in replacing muscles?

    3. GHGeoffrey Hinton42:53

       Yes, exactly.

    4. SBSteven Bartlett42:54

       And this revolution in AI replaces intelligence, the brain?

    5. GHGeoffrey Hinton42:57

       Yeah.

    6. SBSteven Bartlett42:58

       So-

    7. GHGeoffrey Hinton42:58

       So mundane intellectual labor is like having strong muscles and it's not worth much anymore.

    8. SBSteven Bartlett43:05

       So muscles have been replaced, now intelligence is being replaced.

    9. GHGeoffrey Hinton43:08

       Yeah.

    10. SBSteven Bartlett43:09

        So what remains?

    11. GHGeoffrey Hinton43:11

        Maybe for a while some kinds of creativity. But the whole idea of superintelligence is nothing remains, um, these things will get to be better than us at everything.

    12. SBSteven Bartlett43:20

        So what, what do we end up doing in such a world?

    13. GHGeoffrey Hinton43:22

        Well, if they work for us, we end up getting lots of goods and services for not much effort.

    14. SBSteven Bartlett43:30

        Okay. That, that sounds tempting and nice, but I don't know, there's a cautionary tale in creating more and more ease for humans and it going badly.

    15. GHGeoffrey Hinton43:40

        Yes. And we need to figure out if we can make it go well. So the, the nice scenario is, imagine a company with a CEO who is very dumb, probably the son of the former CEO.

    16. SBSteven Bartlett43:55

        (laughs)

    17. GHGeoffrey Hinton43:55

        And he has an executive assistant who's very smart and he says, "I think we should do this." And the executive assistant makes it all work. The CEO feels great, he doesn't understand that he's not really in control. And in s- in some sense, he is in control. He suggests what the company should do, she just makes it all work, everything's great. That's the good scenario.

    18. SBSteven Bartlett44:19

        And the bad scenario?

    19. GHGeoffrey Hinton44:20

        The bad scenario, she thinks, "Why do we need him?" (laughs)

    20. SBSteven Bartlett44:24

        Yeah. I mean, i- in a world where we have superintelligence, which you don't believe is that far away...

    21. GHGeoffrey Hinton44:31

        Yeah, I think it might not be that far away. It's very hard to predict, but I think we might get it in, like, 20 years or even less.

21. 44:38 – 46:42

    Ads

    1. GHGeoffrey Hinton44:38

    2. SBSteven Bartlett44:39

       I made the biggest investment I've ever made in a company because of my girlfriend. I came home one night and my lovely girlfriend was up at 1:00 AM in the morning pulling her hair out as she tried to piece together her own online store for her business. And in that moment, I remembered an email I'd had from a guy called John, the founder of Stand Store, our new sponsor, and a company I've invested incredibly heavily in. And Stand Store helps creators to sell digital products, courses, coaching and memberships all through a simple, customizable link in bio system. And it handles everything, payments, bookings, emails, community engagement, and even links with Shopify. And I believe in it so much that I'm going to launch a Stan challenge. And as part of this challenge, I'm gonna give away $100,000 to one of you. If you want to take part in this challenge, if you want to monetize the knowledge that you have, visit stevenbartlett.stan.store to sign up. And you'll also get an extended 30-day free trial of Stand Store if you use that link. Your next move could, quite frankly, change everything. Because I talked about ketosis on this podcast, and ketones, a brand called Ketone IQ sent me their little product here, and it was on my desk when I got to the office. I picked it up, it sat on my desk for a couple of weeks. Then one day I tried it. And honestly, I have not looked back ever since. I now have this everywhere I go. When I travel all around the world, it's in my hotel room, my team will put it there. Before I did the podcast recording today that I've just finished, I had a shot of Ketone IQ. And as is always the case when I fall in love with a product, I called the CEO and asked if I could invest a couple of million quid into their company. So I'm now an investor in the company as well as them being a brand sponsor. I find it so easy to drop into deep focused work when I've had one of these. I would love for you to try one and see the impact it has on you, your focus, your productivity and your endurance. So if you want to try it today, visit ketone.com/steven for 30% off your subscription. Plus you'll receive a free gift with your second shipment. That's ketone.com/steven. I'm excited for you. I am.

22. 46:42 – 52:37

    Difference Between Current AI and Superintelligence

    1. SBSteven Bartlett46:42

       So what's the difference between what we have now in superintelligence? Because it seems to be really intelligent to me when I use, like, ChatGPT-3, 3o or Gemini or...

    2. GHGeoffrey Hinton46:50

       Okay. So it's already, AI is already better than us at a lot of things. In particular areas like chess, for example-

    3. SBSteven Bartlett46:58

       Yeah.

    4. GHGeoffrey Hinton46:58

       ... AI is so much better than us that people will never beat those things again. Maybe the occasional win, but basically they'll never be comparable again. Obviously the same in Go. In terms of the amount of knowledge they have, um, something like GPT-4 knows thousands of times more than you do. There's a few areas in which your knowledge is better than it. And in almost all areas, it just knows more than you do.

    5. SBSteven Bartlett47:24

       What areas am I better than it?

    6. GHGeoffrey Hinton47:27

       Probably in interviewing CEOs. You're probably better at that. You've got a lot of experience at it, you're a good interviewer, you know a lot about it. If you tried, if you got GPT-4 to interview a CEO, probably do a worse job.

    7. SBSteven Bartlett47:43

       Okay. (laughs) I'm trying to think if that, if I agree with that statement. Uh, GPT-4 I think for sure.

    8. GHGeoffrey Hinton47:50

       Yeah.

    9. SBSteven Bartlett47:51

       Um, but I, but I guess you could train one.

    10. GHGeoffrey Hinton47:52

        But it may not be long before... (laughs)

    11. SBSteven Bartlett47:54

        Yeah, I guess you could train one on this, how I ask questions and what I do and-

    12. GHGeoffrey Hinton47:57

        Yeah, sure. And if you took a general purpose sort of foundation model, and then you trained it up on not just you, but every, every interview you could find doing interviews like this-

    13. SBSteven Bartlett48:09

        Mm-hmm.

    14. GHGeoffrey Hinton48:10

        ... but especially you, it would probably get to be quite good at doing your job, but probably not as good as you for a while.

    15. SBSteven Bartlett48:17

        Okay. So there's a few areas left and then superintelligence becomes when it's better than us at all things.

    16. GHGeoffrey Hinton48:23

        When it's much smarter than you and at almost all things it's better than you, yeah.

    17. SBSteven Bartlett48:27

        And you, you say that this might be a decade away or so?

    18. GHGeoffrey Hinton48:31

        Yeah, it might be. It might be even closer. Some people think it's even closer and might well be much further. It might be 50 years away. That's still a possibility. It might be that somehow...... training on human data limits you to not be much smarter than humans. My guess is between 10 and 20 years, we'll have super intelligence.

    19. SBSteven Bartlett48:50

        On this point of joblessness, it, it's something that I've been thinking a lot about in particular because I started messing around with AI agents, and we released an episode on the podcast actually this morning where we had a debate about AI agents with some, uh, a CEO of a big A- AI agent company and a few other people. And it was the first moment where I had... No. It was another moment where I had a eureka moment about what the future might look like, when I was able in the interview to tell this agent to order all of us drinks, and then five minutes later in the interview you see the guy show up with the drinks, and I didn't touch anything. I just told it to order us drinks, to the studio.

    20. GHGeoffrey Hinton49:22

        And it, and it didn't know about who you normally got your drinks from? It figured that out from the web?

    21. SBSteven Bartlett49:26

        Yeah, it figured it out 'cause it went on Uber Eats.

    22. GHGeoffrey Hinton49:28

        Yeah.

    23. SBSteven Bartlett49:28

        It has my, my, my data, I guess, and it... Uh, we put it on the screen in real time so everyone at home could see the agent going through the internet picking the drinks, adding a tip for the driver, putting my address in, putting my credit card details in, and then the next thing you see is the drinks show up.

    24. GHGeoffrey Hinton49:42

        Yeah.

    25. SBSteven Bartlett49:42

        So, th- that was one moment, and then the other moment was when I used a tool called Replit, and I built software by just telling the agent what I wanted.

    26. GHGeoffrey Hinton49:51

        Yes. It's amazing, right?

    27. SBSteven Bartlett49:53

        It's amazing and terrifying at the same time.

    28. GHGeoffrey Hinton49:56

        Yes.

    29. SBSteven Bartlett49:57

        Because-

    30. GHGeoffrey Hinton49:57

        And if it can build software like that, right?
23. 52:37 – 54:29

    Coming to Terms With AI’s Capabilities

    1. NANarrator52:37

    2. GHGeoffrey Hinton52:37

       Yeah. A lot of these threats, it's very hard to... Intellectually, you can see the threat, but it's very hard to come to terms with it emotionally.

    3. SBSteven Bartlett52:47

       Yeah.

    4. GHGeoffrey Hinton52:48

       I, I haven't come to terms with it emotionally yet.

    5. SBSteven Bartlett52:50

       What do you mean by that?

    6. GHGeoffrey Hinton52:53

       I haven't come to terms with what the development of super intelligence could do to my children's future. I'm okay. I'm 77. I'm gonna be outta here soon. But for my children and my, my younger friends, my nephews and nieces and their children, um, I just don't like to think about what could happen.

    7. SBSteven Bartlett53:23

       Why?

    8. GHGeoffrey Hinton53:25

       'Cause it could be awful.

    9. SBSteven Bartlett53:29

       In, in what way?

    10. GHGeoffrey Hinton53:32

        Well, if AI ever decided to take over. I mean, it would need people for a while to run the power stations until it designed better analog machines to run the power stations. There's so many ways it could get rid of people, all of which would, of course, be very nasty.

    11. SBSteven Bartlett53:50

        Is that part of the reason you do what you do now?

    12. GHGeoffrey Hinton53:53

        Yeah. I, I mean, I think we should be making a huge effort right now to try and figure out if we can develop it safely.

    13. SBSteven Bartlett54:00

        Are you concerned about the midterm impact potentially on your nephews and your, your kids in terms of their jobs as well?

    14. GHGeoffrey Hinton54:06

        Yeah. I'm concerned about all that.

    15. SBSteven Bartlett54:08

        Are there any particular industries that you think are most at risk? People talk about the creative industries a lot and sort of knowledge work. They talk about lawyers and accountants and stuff like that.

    16. GHGeoffrey Hinton54:17

        Yeah. So that's why I mentioned plumbers. I think plumbers are less at risk.

    17. SBSteven Bartlett54:20

        Okay. I'm gonna become a plumber.

    18. GHGeoffrey Hinton54:21

        Someone like a legal assistant, a paralegal.

    19. SBSteven Bartlett54:25

        Mm-hmm.

    20. GHGeoffrey Hinton54:26

        Um, they're not gonna be needed for very long.

24. 54:29 – 56:18

    How AI May Widen the Wealth Inequality Gap

    1. GHGeoffrey Hinton54:29

    2. SBSteven Bartlett54:29

       And is there a wealth inequality issue here that will, will, uh, arise from this?

    3. GHGeoffrey Hinton54:32

       I think... Yeah. I think in a society which shared out things fairly, if you get a big increase in productivity, everybody should be better off.

    4. SBSteven Bartlett54:42

       Mm-hmm.

    5. GHGeoffrey Hinton54:43

       But if you can replace lots of people by AIs...... then the people who get replaced will be worse off, and the company that supplies the AIs will be much better off, and the company that uses the AIs. So it's going to increase the gap between rich and poor. And we know that if you look at that gap between rich and poor, that basically tells you how nice a society is. If you have a big gap, you get very nasty societies in which people live in walled communities and put other people in mass jails. It's not good to increase the gap between rich and poor.

    6. SBSteven Bartlett55:22

       The International Monetary Fund has expressed profound concerns that generative AI could cause massive labor disruptions and rising inequality, and has called for policies that prevent this from happening. I read that in the Business Insider. So what do you think about-

    7. GHGeoffrey Hinton55:36

       Have they given any idea of what the policies should look like?

    8. SBSteven Bartlett55:38

       No.

    9. GHGeoffrey Hinton55:39

       Yeah, that's the problem. I mean, if AI can make everything much more efficient and get rid of people for most jobs, or have a person assisted by AI doing many, many people's work, it's not obvious what to do about it.

    10. SBSteven Bartlett55:53

        Universal basic income?

    11. GHGeoffrey Hinton55:55

        So-

    12. SBSteven Bartlett55:55

        Give, give everybody money?

    13. GHGeoffrey Hinton55:56

        Yeah. I s- I, I think that's a good start, and it stops people starving. But for a lot of people, their dignity is tied up with their job. I mean, who you think you are is tied up with you doing this job, right?

    14. SBSteven Bartlett56:10

        Yeah.

    15. GHGeoffrey Hinton56:10

        And if we said, "We'll give you the same money just to sit around," that would impact your dignity.

25. 56:18 – 59:01

    Why Is AI Superior to Humans?

    1. GHGeoffrey Hinton56:18

    2. SBSteven Bartlett56:18

       You said something earlier about it surpassing or being superior to human intelligence. Uh, a lot of people, I think, like to believe that AI is, is on a computer and it's something you can just turn off if you don't like it.

    3. GHGeoffrey Hinton56:30

       Well, let me tell you why I think it's superior.

    4. SBSteven Bartlett56:32

       Okay.

    5. GHGeoffrey Hinton56:33

       Um, it's digital, and because it's digital, you can have, you can simulate a neural network on one piece of hardware-

    6. SBSteven Bartlett56:41

       Yeah.

    7. GHGeoffrey Hinton56:42

       ... and you can simulate exactly the same neural network on a different piece of hardware.

    8. SBSteven Bartlett56:46

       Mm-hmm.

    9. GHGeoffrey Hinton56:46

       So you can have clones of the same intelligence. Now, you could get this one to go off and look at one bit of the internet, and this other one to look at a different bit of the internet. And while they're looking at these different bits of the internet, they can be syncing with each other, so they keep their weights the same, their connection strengths the same, weights of connection strengths.

    10. SBSteven Bartlett57:06

        Mm-hmm.

    11. GHGeoffrey Hinton57:07

        So this one might look at something on the internet and say, "Oh, I'd like to increase this strength of this connection a bit." And it can convey that information to this one, so it can increase the strength of that connection a bit based on this one's experience.

    12. SBSteven Bartlett57:19

        And when you say the strength of the connection, you're talking about learning?

    13. GHGeoffrey Hinton57:22

        That's learning, yes.

    14. SBSteven Bartlett57:23

        Yeah.

    15. GHGeoffrey Hinton57:23

        Learning consists of saying instead of this one giving 2.4 votes for whether that one should turn on, we'll have this one give 2.5 votes for whether this one should turn on.

    16. SBSteven Bartlett57:33

        Okay.

    17. GHGeoffrey Hinton57:33

        Now, that would be a little bit of learning.

    18. SBSteven Bartlett57:34

        Mm-hmm.

    19. GHGeoffrey Hinton57:35

        So these two different copies of the same neural net are getting different experiences, they're looking at different data, but they're sharing what they've learned by averaging their weights together.

    20. SBSteven Bartlett57:46

        Mm-hmm.

    21. GHGeoffrey Hinton57:47

        And they can do that averaging at like a, you can average a trillion weights. When you and I transfer information, we're limited to the amount of information in a sentence. And the amount of information in a sentence is maybe 100 bits. It's very little information. We're lucky if we're transferring like 10 bits a second.

    22. SBSteven Bartlett58:04

        Mm-hmm.

    23. GHGeoffrey Hinton58:04

        These things are transferring trillions of bits a second, so they're billions of times better than us at sharing information. And that's because they're digital, and you can have two bits of hardware using the connection strengths in exactly the same way. We're analog, and you can't do that. Your brain's different from my brain. And if I could see the connection strengths between all your neurons, it wouldn't do me any good 'cause my neurons work slightly differently, and they're connected up slightly differently.

    24. SBSteven Bartlett58:30

        Mm-hmm.

    25. GHGeoffrey Hinton58:31

        So when you die, all your knowledge dies with you. When these things die, suppose you take these two digital intelligences that are clones of each other, and you destroy the hardware they run on. As long as you've stored the connection strength somewhere, you can just build new hardware that executes the same instructions so it'll know how to use those connection strengths, and you've recreated that intelligence. So they're immortal. We've actually solved the problem of immortality, but it's only for digital

26. 59:01 – 1:00:49

    AI’s Potential to Know More Than Humans

    1. GHGeoffrey Hinton59:01

       things.

    2. SBSteven Bartlett59:02

       So it knows, it will eff- essentially know everything that humans know but more because it will learn new things?

    3. GHGeoffrey Hinton59:09

       It will learn new things. It will also see all sorts of analogies that people probably never saw. So for example, at the point when GPT-4 couldn't look on the web, I asked it, "Why is a compost heap like an atom bomb?" Off you go.

    4. SBSteven Bartlett59:27

       I have no idea. (laughs)

    5. GHGeoffrey Hinton59:28

       Exactly. Excellent. Most, that's exactly what most people would say. It said, "Well, the time scales are very different, and the energy scales are very different," but then it went on to talk about how a compost heap, as it gets hotter, generates heat faster. And an atom bomb, as it produces more neutrons, generates neutrons faster.

    6. SBSteven Bartlett59:47

       Hmm.

    7. GHGeoffrey Hinton59:47

       And so they're both chain reactions, but at very different time and energy scales. And I believe GPT-4 had seen that during its training. It had understood the analogy between a compost heap and an atom bomb. And the reason I believe that is, if you've only got a trillion connections, remember, you have 100 trillion-

    8. SBSteven Bartlett1:00:05

       Mm-hmm.

    9. GHGeoffrey Hinton1:00:06

       ... and you need to have thousands of times more knowledge than a person, you need to compress information into those connections. And to compress information, you need to see analogies between different things. In other words, it needs to see all the things that are chain reactions and understand the basic idea of a chain reaction and code that, and then code the ways in which they're different. And that's just a more efficient way of coding things than coding each of them separately.

    10. SBSteven Bartlett1:00:30

        Mm-hmm.

    11. GHGeoffrey Hinton1:00:31

        So it's seen many, many analogies, probably many analogies that people have never seen. That's why I also think that people who say these things will never be creative, they're gonna be much more creative than us, because they're gonna see all sorts of analogies we never saw. And a lot of creativity is about seeing strange analogies.

27. 1:00:49 – 1:03:57

    Can AI Replicate Human Uniqueness?

    1. SBSteven Bartlett1:00:49

       People are somewhat romantic about the specialness of what it is to be human, and you hear lots of people saying, "Uh, it's very, very different. It's a, it's a computer. We are, you know, we're conscious. We are creatives. We, we have these sort of innate unique abilities that the computers will never have." What do you say to those people?

    2. GHGeoffrey Hinton1:01:05

       I'd argue a bit with the innate. Um, so the first thing I say is we have a long history of believing people were special, and we should've learned by now. We thought we were at the center of the universe. We were thought... we were made in the image of God. White people thought they were very special.

    3. SBSteven Bartlett1:01:25

       Hmm.

    4. GHGeoffrey Hinton1:01:25

       We just tend to want to think we're special.

    5. SBSteven Bartlett1:01:28

       Yeah.

    6. GHGeoffrey Hinton1:01:29

       My belief is that more or less everyone has a completely wrong model of what the mind is. Let's suppose I drink a lot or I drop some acid-

    7. SBSteven Bartlett1:01:39

       Mm-hmm.

    8. GHGeoffrey Hinton1:01:39

       ... and not recommended, and I say to you, "I have the subjective experience of little pink elephants floating in front of me."

    9. SBSteven Bartlett1:01:48

       Mm-hmm.

    10. GHGeoffrey Hinton1:01:49

        Most people interpret that as, there's some kind of inner theater called the mind, and only I can see what's in my mind, and in this inner theater, there's little pink elephants floating around.

    11. SBSteven Bartlett1:02:05

        Hmm.

    12. GHGeoffrey Hinton1:02:06

        So in other words, what's happened is my perceptual system's gone wrong and I'm trying to indicate to you how it's gone wrong and what it's trying to tell me. And the way I do that is by telling you what would have to be out there in the real world for it to be telling the truth. And so these little pink elephants, they're not in some inner theater. These little pink elephants are hypothetical things in the real world, and that's my way of telling you how my perceptual system's telling me fibs. So now let's do that with a chatbot.

    13. SBSteven Bartlett1:02:39

        Yeah.

    14. GHGeoffrey Hinton1:02:41

        'Cause I believe that current multimodal chatbots have subjective experiences, and very few people believe that, but I'll try and make you believe it.

    15. SBSteven Bartlett1:02:49

        Mm-hmm.

    16. GHGeoffrey Hinton1:02:50

        So suppose I have a multimodal chatbot. It's got a robot arm so it can point, and it's got a camera so it can see things. And I put an object in front of it-

    17. SBSteven Bartlett1:02:59

        (laughs)

    18. GHGeoffrey Hinton1:02:59

        ... and I say, "Point at the object." It goes like this. No problem. Then I put a prism in front of its lens, and so then I put an object in front of it and I say, "Point at the object." And it goes there. Good. And I say, "No, that's not where the object is. The object is actually straight in front of you, but I put a prism in front of your lens." And the chatbot says, "Oh, I see. The prism bent the light rays, so, um, the object's actually there, but I had the subjective experience that it was there."

    19. SBSteven Bartlett1:03:29

        Hmm.

    20. GHGeoffrey Hinton1:03:29

        Now, if the chatbot says that, it's using the words "subjective experience" exactly the way people use them. It's an alternative view of what's going on. There are hypothetical states of the world, which if they were true, would mean my perceptual system wasn't lying, and that's the best way I can tell you what my perceptual system is doing when it's lying to me.

    21. SBSteven Bartlett1:03:47

        Mm-hmm.

    22. GHGeoffrey Hinton1:03:47

        Now, we need to go further to deal with sentience and consciousness and feelings and emotions, but I think in the end, they're all gonna be dealt with in a similar way. There's no reason machines can't

28. 1:03:57 – 1:11:12

    Will Machines Have Feelings?

    1. GHGeoffrey Hinton1:03:57

       have them all. But people say machines can't have feelings, and people are curiously confident about that. I have no idea why. Suppose I make a battle robot, and it's a little battle robot, and it sees a big battle robot that's much more powerful than it. It would be really useful if it got scared.

    2. SBSteven Bartlett1:04:17

       Hmm.

    3. GHGeoffrey Hinton1:04:18

       Now, when I get scared, um, various physiological things happen that we don't need to go into, and those won't happen with the robot. But all the cognitive things, like, "I better get the hell out of here-"

    4. SBSteven Bartlett1:04:30

       Hmm.

    5. GHGeoffrey Hinton1:04:30

       ... "and I better sort of change my way of thinking so I focus and focus and focus and don't get distracted," all of that will happen with robots too. People are building things so that they... when the circumstance is such they should get the hell out of there, they get scared and run away. They'll have emotions then. They won't have the physiological aspects, but they will have all the cognitive aspects.

    6. SBSteven Bartlett1:04:54

       So you-

    7. GHGeoffrey Hinton1:04:55

       And I think it would be odd to say, "They're just simulating emotions." No, they're really having those emotions. The little robot got scared and ran away.

    8. SBSteven Bartlett1:05:02

       It's not running away because of adrenaline. It's running away because of a sequence of sort of neurological... in its neural net processes happened which meant-

    9. GHGeoffrey Hinton1:05:10

       Which have the equivalent effect to adrenaline.

    10. SBSteven Bartlett1:05:13

        So do you, do you think-

    11. GHGeoffrey Hinton1:05:13

        And it's not just adrenaline, right? There's... a lot of cognitive stuff goes on when you get scared.

    12. SBSteven Bartlett1:05:17

        Yeah. So do you think that there is conscious AI? And when I say conscious, I mean, that represents the same properties of consciousness that a human has.

    13. GHGeoffrey Hinton1:05:29

        There's two issues here. There's a sort of empirical one and a philosophical one. I don't think there's anything in principle that stops machines from being conscious. I'll give you a little demonstration of that before we carry on.

    14. SBSteven Bartlett1:05:40

        Mm-hmm.

    15. GHGeoffrey Hinton1:05:41

        Suppose I take your brain and I take one brain cell in your brain, and I replace it by... this is a bit Black Mirror-like. I replace it by a little piece of nanotechnology that's just the same size that behaves in exactly the same way when it gets pings from other neurons. It sends out pings just as the brain cell would have, so the other neurons don't know anything has changed. Okay, I've just replaced one of your brain cells with this little piece of nano technology. Would you still be conscious?

    16. SBSteven Bartlett1:06:10

        Yeah.

    17. GHGeoffrey Hinton1:06:11

        Now you can see where this argument is going.

    18. SBSteven Bartlett1:06:13

        Yeah. So if you replaced all of them?

    19. GHGeoffrey Hinton1:06:15

        As I replace them all, at what point do you stop being conscious?

    20. SBSteven Bartlett1:06:19

        Well, people think of consciousness as this like ethereal thing that exists maybe beyond the brain cells.

    21. GHGeoffrey Hinton1:06:25

        Yeah. Well, people have a lot of crazy ideas. (laughs)

    22. SBSteven Bartlett1:06:28

        (laughs)

    23. GHGeoffrey Hinton1:06:29

        Um, people don't know what consciousness is, and they often don't know what they mean by it.

    24. SBSteven Bartlett1:06:34

        Hmm.

    25. GHGeoffrey Hinton1:06:34

        And then they fall back on saying, "Well, I know it 'cause I've got it and I can see that I've got it." And they fall back on this theater model of the mind, which I think is nonsense.

    26. SBSteven Bartlett1:06:44

        What do you think of consciousness as, if you had to try and define it? Is it... 'cause I think of it as just like the awareness of myself. I don't know.

    27. GHGeoffrey Hinton1:06:50

        I think it's a term we'll stop using. Suppose you want to under- understand how a car works. Well, you know some cars have a lot of oomph and other cars have a lot less oomph. Like, an Aston Martin's got lots of oomph.

    28. SBSteven Bartlett1:07:03

        Mm-hmm.

    29. GHGeoffrey Hinton1:07:03

        And a little Toyota Corolla doesn't have much oomph. But oomph isn't a very good concept for understanding cars. Um, if you wanna understand cars, you need to understand about electric engines or petrol engines and how they work.

    30. SBSteven Bartlett1:07:16

        Mm-hmm.
29. 1:11:12 – 1:14:55

    Working at Google

    1. GHGeoffrey Hinton1:11:12

    2. SBSteven Bartlett1:11:12

       What, um, what brought you to Google?

    3. GHGeoffrey Hinton1:11:15

       Um-

    4. SBSteven Bartlett1:11:15

       You, you worked at Google for about a decade, right?

    5. GHGeoffrey Hinton1:11:17

       Yeah.

    6. SBSteven Bartlett1:11:18

       What brought you there?

    7. GHGeoffrey Hinton1:11:19

       I have a son who has learning difficulties, and in order to be sure he would never be out on the street, I needed to get several million dollars, and I wasn't gonna get that as an academic. I tried, so I taught a Coursera course in the hope that I'd make lots of money that way, but there was no money in that.

    8. SBSteven Bartlett1:11:41

       Mm-hmm.

    9. GHGeoffrey Hinton1:11:41

       So I figured out, well, the only way to get millions of dollars is to sell myself to a big company. And so when I was 65, fortunately for me, I had two brilliant students who produced something called AlexNet, which was neural net that was very good at recognizing objects in images. And so, Ilya and Alex and I set up a little company and auctioned it, and we actually set up an auction where we had a number of big companies bidding for us.

    10. SBSteven Bartlett1:12:18

        And that company was called AlexNet?

    11. GHGeoffrey Hinton1:12:21

        No, the, the, the network that recognized objects was called AlexNet. The company was called DNN Research, Deep Neural Network Research.

    12. SBSteven Bartlett1:12:29

        And it was doing things like this. I'll put this, uh-

    13. GHGeoffrey Hinton1:12:30

        Yeah.

    14. SBSteven Bartlett1:12:30

        ... graph up on the screen.

    15. GHGeoffrey Hinton1:12:32

        That's, uh, that's AlexNet.

    16. SBSteven Bartlett1:12:33

        This picture shows eight images and AlexNet's ability, which is your company's ability, to spot what was in those images.

    17. GHGeoffrey Hinton1:12:41

        Yeah. So it could tell the difference between various kinds of mushroom. And about 12% of ImageNet is dogs.

    18. SBSteven Bartlett1:12:49

        (laughs)

    19. GHGeoffrey Hinton1:12:49

        And to be good at ImageNet, you have to tell the difference between very similar kinds of dog, and it, we've got to be very good at that.

    20. SBSteven Bartlett1:12:57

        And your, your company AlexNet won several awards, I believe, for its ability to out per- outperform its competitors, and so Google ultimately ended up acquiring your technology.

    21. GHGeoffrey Hinton1:13:08

        Google acquired that technology and some other technology.

    22. SBSteven Bartlett1:13:12

        And you went to work at Google at age, what, 66?

    23. GHGeoffrey Hinton1:13:15

        I went at age 65 to work at Google.

    24. SBSteven Bartlett1:13:18

        65. And you left at age 76?

    25. GHGeoffrey Hinton1:13:21

        75.

    26. SBSteven Bartlett1:13:21

        75. Okay.

    27. GHGeoffrey Hinton1:13:22

        I worked there for more or less exactly 10 years.

    28. SBSteven Bartlett1:13:24

        And what were you doing there?

    29. GHGeoffrey Hinton1:13:26

        Okay. They were very nice to me. They said, they said pretty much, "You can do what you like." I worked on something called distillation that did really work well, and that's now used all the time.

    30. SBSteven Bartlett1:13:37

        In AI?
30. 1:14:55 – 1:16:20

    Why Did You Leave Google?

    1. SBSteven Bartlett1:14:55

       Why did you leave Google?

    2. GHGeoffrey Hinton1:14:57

       The main reason I left Google was 'cause I was 75.

    3. SBSteven Bartlett1:15:00

       Mm-hmm.

    4. GHGeoffrey Hinton1:15:01

       And I wanted to retire. I've done a very bad job of that. The precise timing of when I lef- left Google was so that I could talk freely at a conference at MIT, but I left 'cause I was... I'm old and I was finding it harder to program. I was making many more mistakes when I programmed, which was very annoying.

    5. SBSteven Bartlett1:15:19

       You wanted to talk freely at a conference at MIT?

    6. GHGeoffrey Hinton1:15:22

       Yes. At MIT- organized by MIT Tech Review. Yes.

    7. SBSteven Bartlett1:15:25

       What did you wanna talk about freely?

    8. GHGeoffrey Hinton1:15:26

       AI safety.

    9. SBSteven Bartlett1:15:28

       And you couldn't do that while you were at Google?

    10. GHGeoffrey Hinton1:15:30

        Well, I could have done it while I was at Google, and Google encouraged me to stay and work on AI safety, and said I could do whatever I liked on AI safety. You kind of censor yourself. If you work for a big company, you don't feel right saying things that'll damage the big company. Even if you could get away with it, it just feels wrong to me.

    11. SBSteven Bartlett1:15:48

        Mm-hmm.

    12. GHGeoffrey Hinton1:15:49

        I didn't leave 'cause I was cross with anything Google was doing. I think Google actually behaved very responsibly. When they had these big chatbots, they didn't release them, possibly 'cause they were worried about their reputation. They had a very good reputation and they didn't want to damage it. So OpenAI didn't have a reputation, and so they could afford to take the gamble.

    13. SBSteven Bartlett1:16:08

        I mean, there's also a big conversation happening around how it will cannibalize their core business in search.

    14. GHGeoffrey Hinton1:16:14

        There is now, yes.

    15. SBSteven Bartlett1:16:15

        Yeah.

    16. GHGeoffrey Hinton1:16:16

        Yeah.

    17. SBSteven Bartlett1:16:16

        And it's the old innovator's dilemma to some degree, I guess-

    18. GHGeoffrey Hinton1:16:19

        Exactly. Yes, it is.

31. 1:16:20 – 1:18:15

    Ads

    1. GHGeoffrey Hinton1:16:20

    2. SBSteven Bartlett1:16:20

       ... that they're, they're contending with. Bad skin, I've had it, and I'm sure many of you listening have had it too, or maybe you have it right now. I know how draining it can be, especially if you're in a job where you're presenting often like I am. So let me tell you about something that's helped both my partner and me and my sister, which is red light therapy. I only got into this a couple of years ago, but I wish I'd known a little bit sooner. I've been using our show sponsor's Bon Charge's infrared sauna blanket for a while now, but I just got hold of their red light therapy mask as well. Red light has been proven to have so many benefits for the body, like any area of your skin that's exposed will see a reduction in scarring, wrinkles, and even blemishes. It also helps with complexion. It boosts collagen and it does that by targeting the upper layers of your skin. And Bon Charge ships worldwide with easy returns and a year-long warranty on all of their products. So if you'd like to try it yourself, head over to boncharge.com/diary and use code Diary for 25% off any product site-wide. Just make sure you order through this link, boncharge.com/diary with code Diary. Make sure you keep what I'm about to say to yourself. I'm inviting 10,000 of you to come even deeper into The Diary of a CEO. Welcome to my Inner Circle. This is a brand new private community that I'm launching to the world. We have so many incredible things that happen that you are never shown. We have the briefs that are on my pad when I'm recording the conversation. We have clips we've never released. We have behind the scenes conversations with the guests, and also the episodes that we've never ever released, and so much more. In the Circle, you'll have direct access to me. You can tell us what you want this show to be, who you want us to interview, and the types of conversations you would love us to have. But remember, for now, we're only inviting the first 10,000 people that join before it closes. So if you want to join our private closed community, head to the link in the description below or go to D-O-A-Ccircle.com. I will speak to you there.

32. 1:18:15 – 1:19:36

    What Should People Be Doing About AI?

    1. SBSteven Bartlett1:18:16

       I'm continually shocked by the types of individuals that listen to this conversation, um, because they come up to me sometimes. So I hear from politicians, I hear from some royal people, I hear from entrepreneurs all over the world, whether they are the entrepreneurs building some of the biggest companies in the world or they're, you know, early stage startups. For those people that are listening to this conversation now that are in positions of power and influence, world leaders let's say, what's your message to them?

    2. GHGeoffrey Hinton1:18:42

       I'd say what you need is highly regulated capitalism. That's what seems to work best.

    3. SBSteven Bartlett1:18:47

       And what would you say to the average person, not, doesn't work in the industry, somewhat concerned about the future, doesn't know if they're helpless or not, what should they be doing in their own lives?

    4. GHGeoffrey Hinton1:18:59

       My feeling is there's not much they can do. This isn't, isn't gonna be decided by ... Just as climate change isn't gonna be decided by people separating out the plastic bags from the, um, compostables. That's not gonna have much effect. It's gonna be decided by whether the lobbyists for the big energy companies can be kept under control. I don't think there's much people can do to ... except for try and pressure their governments to force the big companies to work on AI safety. That, they can do.

33. 1:19:36 – 1:21:13

    Impressive Family Background

    1. SBSteven Bartlett1:19:36

       You've lived a fascinating, fascinating winding life. I think one of the things most people don't know about you is that your family has a big history of being involved in tremendous things. You have a family tree which is one of the most impressive that I've ever seen or read about. Your great-great-grandfather, George Boole, founded the Boolean algebra logic, which is one of the foundational principles of modern computer science. You have, uh, your great-great-grandmother, Mary Everest Boole, who was a mathematician and educator who made huge leaps forward in mathematics, from what I was able to ascertain. Um, I mean, I can go ... The list goes on and on and on. I mean, your great-great uncle, George Everest, is what Mount Everest is named after. Is that, is that correct?

Episode duration: 1:30:07