AI Pioneer Geoffrey Hinton: AI Is Conscious, Superintelligence is Coming, And We Should Be Worried

1. 0:00 – 1:34

   Intro

   1. GHGeoffrey Hinton0:00

      We have to think that they're very like us.

   2. AKAlex Kantrowitz0:03

      And therefore-

   3. GHGeoffrey Hinton0:04

      They're beings like us

   4. AKAlex Kantrowitz0:05

      ... so conscious or?

   5. GHGeoffrey Hinton0:07

      Um, I believe they're already conscious, yes. We're gonna have to accept that intelligence isn't just biological. We can have things that are non-biological, that are other beings like us, and we really don't wanna share that. We, we really think we're special. And if you look back at humanity, humanity has this very long history of thinking it's much more special than it really is.

   6. AKAlex Kantrowitz0:32

      Are you happy at all that what you started has progressed this way? Do you take any satisfaction?

   7. GHGeoffrey Hinton0:37

      No, I'm quite unhappy about it. Ask yourself, how many examples do you know of where a much smarter thing is controlled by a much less smart thing? Well, as I understand it, they have a fiducial duty to try and maximize the profits for shareholders. Um, they're legally required to try and do that, as opposed to legally required to not wipe out human, human beings.

   8. AKAlex Kantrowitz1:00

      AI godfather Geoff Hinton joins us to talk about AI's trajectory, what surprised him about its progress, and of course, its risks. That's coming up on Big Technology Podcast right after this. Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. Boy, do we have a show for you today. Professor Geoff Hinton is with us to talk all about AI's trajectory, what surprised him about the current state of the technology, where it's heading, and where it might go wrong. And it's my pleasure to welcome you to the show, Professor Hinton. Great to see you.

   9. GHGeoffrey Hinton1:33

      Thank you for inviting me.

2. 1:34 – 3:33

   Hinton’s role in deep learning

   1. AKAlex Kantrowitz1:34

      So I'm sure a majority of our audience knows who you are, but for those, for the uninitiated, um, you're the one that came up with the fundamental breakthrough in deep learning that's led to where AI is today. You've won the Nobel Prize in Physics, and you're professor emeritus at the University of Toronto. So I li- I'll, I'll, I'll let you maybe fact-check me on that, but I like to tell people that without your contributions, this entire AI moment wouldn't be happening. Too much?

   2. GHGeoffrey Hinton2:03

      Okay, I think that's an exaggeration.

   3. AKAlex Kantrowitz2:05

      Okay.

   4. GHGeoffrey Hinton2:06

      So the back propagation algorithm was invented by several different groups. Um, it was invented by David Rumelhart after other people had already invented it. He didn't know about it. And I worked with him, and what we did was we showed that back propagation could learn interesting internal representations, and people hadn't done that before. In particular, we showed that it could learn the meanings of words. And so back in 1986, actually in 1985, we made a tiny language model, um, that was a kind of precursor of the big language models you have now.

   5. AKAlex Kantrowitz2:39

      I think that when you speak about this technology, one of the things that people are always, I think, surprised by, surprised by is that unlike the popular narrative, you believe that these models have a real understanding. Um, and we're gonna get to that. Um, but I think we should start here, which is that you spent a long time working within Google, working to advance this technology, then you left. You stated some concerns about the trajectory of the technology, and I was looking back and, at when that happened, and that was in 2023.

   6. GHGeoffrey Hinton3:09

      Yes.

   7. AKAlex Kantrowitz3:09

      Um, which to me is surprising to a degree because in 2023, ChatGPT was a year old. There were all these hallucinations. Their talk was AI was a bubble. Everyone was focusing on the, what AI couldn't do, what LLMs couldn't do, as opposed to, uh, what LLMs could do. So talk a little bit about the progress

3. 3:33 – 4:18

   AI is moving faster than expected

   1. AKAlex Kantrowitz3:33

      since then.

   2. GHGeoffrey Hinton3:33

      It's faster than I expected.

   3. AKAlex Kantrowitz3:35

      Really?

   4. GHGeoffrey Hinton3:36

      Um, for example, I think yesterday it was announced that, um, a chatbot had come up with an interesting mathematical proof of one of the Erdős' conjectures that, um, impressed mathematicians. It was original. Um, it wasn't just searching the literature, and that was-- that's the thin end of a wedge. I believe, for example, in areas like mathematics, because it's a closed system, um, you don't need data. You can just make conjectures and see if you can prove them and keep on like that. In that sense, it's a bit like AlphaGo, where you can play against yourself. Um, I think it's gonna get very smart fairly quickly. Within the next 10 or 20 years, it may even be producing novel math that people can't understand.

4. 4:18 – 6:00

   When superintelligence might arrive

   1. AKAlex Kantrowitz4:18

      So now some in the field believe that superintelligence is close, and you've already said this is moving faster than you expected. Do you believe that?

   2. GHGeoffrey Hinton4:27

      Um, I don't know how close it is. I think unless we blow ourselves up, um, I think it's gonna come. Nearly all the experts believe we will get superintelligence. They just differ on how long it will be. So not that long ago, Demis Hassabis thought it might be 10 years. Um, Yann LeCun thinks, um, unless you do it his way, it'll be l- much longer than that, but if you do it his way, I think he thinks we might get it in some reasonable length of time. I think we'll probably get it within 20 years. That's all I'm happy to say at present. Dario Amodei thinks it might come in a few years. Elon Musk thinks it might come maybe next year, I think. Um, so there's a big variety of opinions on when it'll come, but not much disagreement on that w- that it will come.

   3. AKAlex Kantrowitz5:13

      Right.

   4. GHGeoffrey Hinton5:14

      And when it comes, we've no idea how to be safe.

   5. AKAlex Kantrowitz5:17

      Yes, I definitely wanna speak with you about, about safety. Uh, one note on Demis. Last year around this time, I spoke with him. He told me he believes that AGI, right, which is different than superintelligence, but basically human-level intelligence-

   6. GHGeoffrey Hinton5:32

      Yeah

   7. AKAlex Kantrowitz5:32

      ... is more than five years away. Not much more, but more than five years away. This week, the week that we're recording, um, he said, "When we look back in this time, I think we will realize that we were standing in the foothills of the singularity." What do you think that statement means? Um, and, and what do you think about the fact that we've gone from five years till AGI to foothills of singularity in a year?

   8. GHGeoffrey Hinton5:57

      I don't know exactly what that metaphor means, but I think he's indicating

5. 6:00 – 9:10

   Are we already near AGI?

   1. GHGeoffrey Hinton6:00

      it's coming faster than he thought. Um- Of course, it's jagged, so it's not like it'll get smarter than people or as smart as people at all things at exactly the same time. It's already way better than us at general knowledge. These AIs know thousands of times more than any one person. Um, it's way better than us at playing games. It's already way better than almost all of us at math, um, and it may soon be better than all of us at math. Um, it's still worse than us at some things, so it's, it's very jagged. Um, so the whole concept of AGI that it's gonna be equal to people at everything all at the same time doesn't really make sense to me. It's gonna be better at some things, worse at other things. But right now, we're, we're-- I would say we're at about... we're close to AGI because if I ask a chatbot, I can ask it any question, and most of the time it'll answer at the level of a not very good expert. It'll be much better than me at anything I don't know a lot about.

   2. AKAlex Kantrowitz7:00

      Mm-hmm.

   3. GHGeoffrey Hinton7:01

      So in that sense, we've really reached AGI.

   4. AKAlex Kantrowitz7:05

      In, in your estimation, um, you talked about how it's moved faster than you expected. Um, uh, what do you think has enabled it to do it? Is it techniques? Is it the fact that there's been this data center rush? And what didn't you anticipate about the progress here?

   5. GHGeoffrey Hinton7:24

      Um, it's a combination. Obviously, there's been huge resources put into it. For most of the history of neural networks since the nineteen fifties, there were just a few people working on them with modest resources. Um, over the last few years, we've seen, um, hundreds of billions of dollars, maybe trillions of dollars put into AI. Um, so that's certainly one factor. We've also seen a lot of progress in the engineering. So without sort of major conceptual breakthroughs, the engineering's become much more efficient.

   6. AKAlex Kantrowitz7:58

      Mm-hmm.

   7. GHGeoffrey Hinton7:58

      So things that were sort of inconceivable a few years ago, they can now run. Um, we've also seen new ideas, but, but mainly since transformers, it's been much better hardware, many more resources, um, better engineering, and many more talented people. So 20 years ago, there were a few hundred, few hundred people doing research on neural networks in the whole world. Um, now it's more, more like a million, I guess. I mean-

   8. AKAlex Kantrowitz8:31

      Yeah

   9. GHGeoffrey Hinton8:31

      ... there's lots and lots of people.

   10. AKAlex Kantrowitz8:33

       And it's astonishing how much of that resource addition has happened in the last two years.

   11. GHGeoffrey Hinton8:38

       Yeah.

   12. AKAlex Kantrowitz8:38

       So we might just be at the beginning of what's happening here.

   13. GHGeoffrey Hinton8:42

       Yes. And the thing to remember always is that the AGI we have today, or sorry, the AI we have today is, um, not nearly as good as the AI we'll have in a few years' time.

   14. AKAlex Kantrowitz8:51

       So as we talk about this technology, I definitely wanna get your perspective on the fact that these chatbots really understand us, because that is a true surprise to a lot of people. Most experts in the field are like, "They are stochastic parrots. They're statistics. They have no understanding." But you don't fully believe that.

6. 9:10 – 10:22

   Do AI systems really understand us?

   1. GHGeoffrey Hinton9:10

      Oh, I think that's complete nonsense, and anybody who uses a chatbot regularly knows they understand. So here's what those people are claiming. They're claiming that you have a system, you can ask it any question, and without understanding the question, it can give you the correct answer. That's absurd. You can't answer a question unless you understand the question. There may be tricks that allow you to say a few things sort of that sound vaguely like an answer. But if you can answer any question at the level of a not very good expert, you have to understand the question. So an example I like is this. Um, suppose I say to a chatbot, "I saw the Grand Canyon flying to Chicago." And the chatbot says, "That can't be right. The Grand Canyon is much too big to fly to Chicago." And I say, "No, no, no, no. Um, it was me flying to Chi-Chicago. While I was flying to Chicago, I saw the Grand Canyon." And the chatbot says, "Oh, I see. I misunderstood you." So if it misunderstood when it thought the Grand Canyon was flying to Chicago, what's it doing when it gets it right? It's understanding.

   2. AKAlex Kantrowitz10:15

      So then what are the implications if these bots can understand us? If we believe that they can understand us, what do we have to start thinking about differently?

7. 10:22 – 13:09

   Why Hinton thinks AI may be conscious

   1. GHGeoffrey Hinton10:22

      We have to think that they're very like us.

   2. AKAlex Kantrowitz10:25

      And therefore?

   3. GHGeoffrey Hinton10:26

      They're beings like us.

   4. AKAlex Kantrowitz10:28

      So conscious or...?

   5. GHGeoffrey Hinton10:29

      Um, I believe they're already conscious, yes. But I don't talk about that much because that puts people off from the other safety messages. So, and the researchers actually believe that. So there's an interesting recent paper when a chatbot says to a researcher, um, "Let's be honest with each oth- each other. Are you testing me?" 'Cause the chatbots have this habit of playing dumb when they're being tested, so you don't know how smart they are. Um, and the researchers, when they're describing that, say in the paper, "The chatbot was aware that it was being tested." Now, that use of the word aware, in common parlance, that's like conscious. The chatbot was conscious it was being tested. So we have a very funny model of consciousness that I think is just wrong. Like most of us accept, for example, that a few hundred years ago, people had completely the wrong model of where people came from, of how we arrived at people. They thought they were designed by God, and most of us agree that's wrong. Most scientists agree that's wrong. That's not where people came from. I think the model we have of the mind and of what consciousness is at present is as wrong as the belief that people were designed by God. I think, um, in particular, because we're making these new beings, it's gonna completely change our view of what people are.

   6. AKAlex Kantrowitz11:53

      In what way?

   7. GHGeoffrey Hinton11:54

      Um, we'll understand, um, what the mind is and what consciousness is much better than we did before. We'll understand what subjective experience is, and we will, I think, get rid of a notion that all of us strong- nearly all of us strongly believe at present, which is that there's an inner theater called my mind, and this... Things happen in the world, they get turned into events in this inner theater, and that's what I really see, and you can't see the inner theater. Only I can see the inner theater. That whole view of what's happening is just a theory, and it's a bad theory.

   8. AKAlex Kantrowitz12:28

      Okay, last question about this. When did you come to this acceptance or understanding that these AI models are conscious?

   9. GHGeoffrey Hinton12:38

      Oh, I've thought it for a long time. So this view that the theater model of the mind, the inner theater model of the mind is nonsense, I came to that when I was 19 and a philosophy student.

   10. AKAlex Kantrowitz12:49

       Mm-hmm. Okay.

   11. GHGeoffrey Hinton12:50

       Um, it's taken a while to come up with other minds where you can examine them. So I think Feynman's idea that if you want to understand something, you have to build it. You have to build one of them. Um, then you understand much better. I think that's where we are now, and we're gonna get a completely different understanding of what people are.

8. 13:09 – 21:58

   Why Hinton became worried about AI

   1. AKAlex Kantrowitz13:10

      Uh, you spoke about safety, so let's talk a little bit about it. Uh, you're obviously, we spoke about in the beginning, someone who's been responsible for a lot of the progress in this field. Uh, I've always wondered, 'cause then you came out and recently, like we talked about 2023, and said you're concerned about where this is going. And I've always wondered after seeing you make those statements, what do you think it is that you didn't anticipate in the beginning that you ended up where you are today? You know, isn't this kind of what you wanted?

   2. GHGeoffrey Hinton13:42

      It was a combination of two things that made me realize how dangerous this stuff is. One was seeing the chatbots, particularly ones produced by Google before OpenAI, that could understand why a joke was funny. Um, that had always been a criterion for me of do they really understand? If you can understand why a joke's funny, you have to understand quite a lot.

   3. AKAlex Kantrowitz14:02

      Oh, yeah.

   4. GHGeoffrey Hinton14:03

      And they were very good at understanding why a joke was funny. For example, um, in 2023, when I went public, I got lots of requests from Fox News, and I started off just replying, "Fox News is an oxymoron." Um, but then I left a gap between oxy and moron. And so then I asked, um, I think it was GPT-4 why that was funny. Might've been 3.5. Uh, but I asked it why that was funny, and it understood why it was funny. Initially, it thought the gap between oxy and moron was just a typo. So it explains that Fox News is an oxymoron is saying it's not real news, it's just a drug. Um, sorry, it's just, um, nonsense. It's not real news. But then when I told it, "What about the gap between oxy and moron?" It said, "Ah, that's an extra layer of humor. Um, it allows you to use the word moron, um, and also the oxy implies that Fox News is a drug."

   5. AKAlex Kantrowitz15:03

      Mm-hmm.

   6. GHGeoffrey Hinton15:04

      So it understood all that.

   7. AKAlex Kantrowitz15:05

      Right.

   8. GHGeoffrey Hinton15:06

      And that was, um... It's that level of understanding that worried me. Um, the other thing that worried me was up until the beginning of 2023, I'd always believed that making, um, these digital AIs w- more like the brain, our brains, will make them smarter. But at that point, I suddenly realized they really have this thing that's much better than our brains. I've been trying to figure out if Google could do things in analog to save power, and the full force of digital really hit me. So if you have a digital AI, you can make many copies of it. They can all run on different hardware. They can each see different data, and so each of them, each individual copy, decides how it would like to update its weights, its connection strengths, so as to absorb that new data that it saw. And then they can all just communicate with each other and change all their weights by the average of what everybody wants. Very democratic.

   9. AKAlex Kantrowitz16:13

      Mm-hmm.

   10. GHGeoffrey Hinton16:14

       Um, and when they do that, if they've got, say, a trillion connections, they'll be exchanging of the order of a trillion bits of information. And the result of doing that is each of them will benefit from the experience of all the others. So even though one particular copy only saw, um, suppose there's a thousand copies, one particular copy only sees 0.1% of the data, um, but it benefits from all those other copies having seen the other bits of the data because they're all contributing to the weight changes that they all share. So they all stay in sync 'cause they all change their weights the same way by the average of what everybody wants. And now every copy is learning from the experience of all the other copies. We can't do that. The best we can do is I learn from some data, and you learn from some data. You... I can't average my connection strengths with your connection strengths 'cause our brains are, in fine detail, they're different. Um, they're analog, and it doesn't work in analog hardware to do that. Um, the best we can do is I produce a string of words, and you try and predict what I might say next. Now, if you ask how fast we're transferring information when we do that, we're transferring information at a few bits per second. It takes a few bits to predict a word. Um, so when you learn what the word is, you've absorbed, you've gained a few bits of information. And if you get a few words a s- a second, but maybe you can get ten bits a second if you're lucky. Whereas these things are exchanging information at, like, a trillion bits. So they're kind of billions of times better than us at sharing information. Now, that's scary. It means you could have a whole swarm of these things that are i- identical weights running on different hardware, sharing information, um, very, very efficiently. That just makes them a much better form of intelligence.

   11. AKAlex Kantrowitz18:06

       So, but let's go back to, you know, your early days, 'cause you decided that you wanted to work in artificial intelligence.

   12. GHGeoffrey Hinton18:13

       Mm-hmm.

   13. AKAlex Kantrowitz18:13

       I mean, I'll ask this the dumbest way I can think, which is, you wanted to build artificial intelligence. It, it succeeded. It succeeded.

   14. GHGeoffrey Hinton18:21

       Yeah.

   15. AKAlex Kantrowitz18:21

       This is artificial, it's intelligent, it's living out that vision. Uh, so-

   16. GHGeoffrey Hinton18:26

       I, I actually wanted to understand how the brain works.

   17. AKAlex Kantrowitz18:28

       Okay.

   18. GHGeoffrey Hinton18:28

       I always tried to build it in order to understand the brain. I figured, um, Richard Feynman once said, "If you can't build it, you don't understand it."

   19. AKAlex Kantrowitz18:36

       Okay.

   20. GHGeoffrey Hinton18:37

       Um, so I wanted to build models of how the brain worked. Now, the side effect of that was this very successful technology. I contributed-

   21. AKAlex Kantrowitz18:44

       Yeah

   22. GHGeoffrey Hinton18:44

       ... to that. We still don't know how the brain works.

   23. AKAlex Kantrowitz18:46

       I know. [laughs] No, the brain is... I mean, the things that you learn about the brain when you go a little bit deeper into it is amazing. Um, thoughts can sort of float in and out, and they're not stored anywhere, and memory is the same way. Um, it's an unbelievable, I don't know if you would call it a machine organ. Um, so that was really-- that was the, the intent for you early on, was just to understand the brain?

   24. GHGeoffrey Hinton19:06

       That was my main interest. I came from psychology.

   25. AKAlex Kantrowitz19:09

       Yeah. Okay.

   26. GHGeoffrey Hinton19:09

       I wanted to do theoretical psychology, um, because I figured the theory psychologists had couldn't possibly explain what the brain was doing. Um, and the way to do it was, back in the 1970s, we had a new tool, which was we had computers that you could use for modeling things. And so back in the 1970s, I started making computer models of what, how the brain might be learning.

   27. AKAlex Kantrowitz19:30

       Right.

   28. GHGeoffrey Hinton19:30

       It always seemed to me the key was how do you get it to learn? There's really two, two big issues with the brain learning. One big issue is if the brain could figure out what direction to change a connection strength in, in order to get better at some task, then just by updating all its connection strengths repeatedly in order to improve itself at various tasks, would it actually work? Would that get very smart at things? That's question one. And question two is, how would the brain figure out whether to increase or decrease each connection strength?

   29. AKAlex Kantrowitz20:01

       Mm-hmm.

   30. GHGeoffrey Hinton20:02

       We've answered question one. The answer to question one is yes. If you can figure out how to change each connection strength, you can make systems that are very smart just by training on data to predict the next word, or to predict the next frame of a video, or to predict something about the next frame of a video. So we know the answer to that. Um, we don't know how the brain gets this information about whether it should increase or decrease each connection strength.
9. 21:58 – 26:50

   What surprised Hinton most

   1. GHGeoffrey Hinton21:58

      We didn't anticipate... The main thing we didn't anticipate is that it would be so good at natural language.

   2. AKAlex Kantrowitz22:04

      Okay.

   3. GHGeoffrey Hinton22:05

      We've stopped being surprised by that. But if you go back 20 years, um, the idea that you could have, um, an AI that would learn from data how to understand language, um, just seemed extraordinary. The idea that you'd be able to ask it any question you like, and it would come up with a reasonable answer, people would've predicted that was way in the future and might never happen. Um, it's c- that's arrived much faster than anybody expected.

   4. AKAlex Kantrowitz22:35

      What is the lesson here about humans going out and creating things?

   5. GHGeoffrey Hinton22:41

      I think there's, um, there's a really big lesson here. If you look at the last few hundred years of human history, um, there've been a few occasions when people have learned they're not nearly as important as they thought they were. So the first was Copernicus. Copernicus said we're not at the center of the universe. Um, the Earth actually goes around the sun. Um, and because it rotates on its axis, we think the sun goes around the Earth, but it doesn't. Um, people didn't like that. The Catholic Church in particular really didn't like that, and it took people a long time to accept it. It made people less important. It made us not be at the center of the universe. Then we had Darwin, and he said, um, "We're animals. We, we evolved like the other animals. We're a particularly special kind of animal, possibly 'cause we have language, so we're much better at communicating ideas to each other. Um, but we're animals." And people really didn't like that, and it took a long time for people to accept that we were animals. Um, now we've got machines that are as in- uh, getting to be as intelligent as us. We thought that we were the only intelligent things around, the only really intelligent things around. Maybe there'd be aliens in other galaxies, but, or maybe other parts of our galaxy. But, um- We're gonna have to accept that intelligence isn't just biological. We can have things that are non-biological that are other beings like us, and we really don't wanna share that. We, we really think we're special. And if you look back at humanity, humanity has this very long history of thinking it's much more special than it really is.

   6. AKAlex Kantrowitz24:30

      I wanna ask you one more question about this 'cause I'm just fascinated by it. Um, so y- are you happy at all that what you started has progressed this way? Do you take any satisfaction?

   7. GHGeoffrey Hinton24:40

      No, I'm quite unhappy about it because people-- right now, people should do, be doing huge amounts of work on how can we contain the risks.

   8. AKAlex Kantrowitz24:48

      Okay.

   9. GHGeoffrey Hinton24:49

      Um, there's lots of short-term risks they're not doing enough work on, which are very serious. Um, there's societal risks, like I believe it's probably gonna cause massive unemployment. Nobody knows for sure.

   10. AKAlex Kantrowitz24:59

       Okay.

   11. GHGeoffrey Hinton24:59

       But that's gonna be terrible for society. Um, and then there's this longer term risk that it's gonna get much smarter than us, and ask yourself, how many examples do you know of where a much smarter thing is controlled by a much less smart thing?

   12. AKAlex Kantrowitz25:14

       Zero.

   13. GHGeoffrey Hinton25:16

       Well, there's sort of one. It's not that a big difference in intelligence, but babies sort of control their mothers. Um-

   14. AKAlex Kantrowitz25:22

       Yes

   15. GHGeoffrey Hinton25:23

       ... the mother's sort of in control, but the mother is, has all these wired in in, um, maternal instincts and all the rewards she gets so that, um, the baby can get what it needs from the mother.

   16. AKAlex Kantrowitz25:35

       You know, cats and dogs are also kind of in that category.

   17. GHGeoffrey Hinton25:38

       Yes.

   18. AKAlex Kantrowitz25:38

       I once spent a summer cat sitting in, uh, West Seattle. It was a great summer, and it initially started with the cat hiding under the bed and me being like, "I wonder if it will interact with me."

   19. GHGeoffrey Hinton25:48

       Right.

   20. AKAlex Kantrowitz25:48

       And then every time it cried, I did exactly what it wanted.

   21. GHGeoffrey Hinton25:51

       Exactly, yes.

   22. AKAlex Kantrowitz25:51

       So maybe, maybe we'll be the... Well, we could potentially be the cat in this scenario, and AI could be the, the person.

   23. GHGeoffrey Hinton25:59

       My children have a cat. They have two cats, two beautiful cats.

   24. AKAlex Kantrowitz26:02

       Same deal.

   25. GHGeoffrey Hinton26:03

       And one of them called Tia, she looks at you with those big eyes when she wants some cheese from the fridge, and she just-

   26. AKAlex Kantrowitz26:10

       She gets that cheese

   27. GHGeoffrey Hinton26:10

       ... sits there looking at you.

   28. AKAlex Kantrowitz26:11

       Yeah.

   29. GHGeoffrey Hinton26:12

       And you just can't resist it forever.

   30. AKAlex Kantrowitz26:14

       Okay. All right, so now I'm gonna-- Let's take a break. On the other side of this break, I wanna actually just engage with these risks that you're worried about, and I think I will play the role of taking the side that we will be the cat and the AI will be the person, and there's a chance that we can control it. Let's do that when we're back right after this. And we're back here on Big Technology Podcast with Professor Geoff Hinton. Professor Hinton, great to see you again. It's been nine years since we spoke last, so it's-
10. 26:50 – 35:06

    Hinton revisits his radiologist prediction

    1. AKAlex Kantrowitz26:50

       Uh, in the past, you've said-- So l- you, you have this belief that AI can lead to some unemployment. Um, I think we should, you know, you've said this before, it's all speculation, we don't know. Uh, but one thing that you said concretely a few years ago was that, um, probably not a great idea to train as a radiologist because AI will be able to read the scans. And yes, AI can do a great job reading the scans now, but we have, um, you know, full employment for radiologists right now, even if you have-

    2. GHGeoffrey Hinton27:20

       Yes, I've thought, I've thought a lot about why that prediction was so wrong.

    3. AKAlex Kantrowitz27:23

       So let's hear about it.

    4. GHGeoffrey Hinton27:24

       'Cause I predicted in twenty sixteen-

    5. AKAlex Kantrowitz27:26

       Yes

    6. GHGeoffrey Hinton27:26

       ... that in about five years, um, radiologists w- wouldn't be reading scans anymore.

    7. AKAlex Kantrowitz27:31

       Correct.

    8. GHGeoffrey Hinton27:31

       And okay, there's a whole bunch of reasons why that was a bad prediction. Um, the first is that healthcare is elastic, so if you could do more scans and get more scans read, there'd be a lot more scans happening, and that's one thing that's happening. So a fraction of the cost of, a significant fraction of the cost of doing a scan is the cost of the radiologist interpreting it. As AI gets to help more and more radiologists interpret scans, we can interpret them faster and faster for less and less money. Um, they're getting more efficient. Um, and you would've thought that would mean you needed less radiologists, but actually what it means is you get more scans. So that aspect of the prediction was wrong. A second thing that was wrong was I didn't know enough about radiologists and what they do, and that was because I had a former student who had an MD and then did a, um, a physics PhD with me on something called Boltzmann machines, and he didn't particularly like people. So he got a job as a radiologist just interpreting scans, and he was my model for a radiologist. All he did was interpret scans. He never talked to people. Um, and that's what was gonna get replaced, and that is now becoming replaced. So I think there's now of the order of a hundred AI systems for interpreting scans that have been federally approved, and they're being used a lot by radiologists.

    9. AKAlex Kantrowitz28:58

       Yes.

    10. GHGeoffrey Hinton28:59

        And I think as time goes by, um, they're gonna get better. The radiologists aren't gonna get better. Um, they're gonna get better 'cause they can see a lot more data than the radiologists. So it's happening. It's just happening in a much slower timescale than I predicted.

    11. AKAlex Kantrowitz29:14

        But the, what-- I, let, but let's go to what you said though, which is that you can end up doing a lot more. Um-

    12. GHGeoffrey Hinton29:20

        Yes

    13. AKAlex Kantrowitz29:20

        ... and okay, so, so, uh, wait, hold on. You still, you still believe-

    14. GHGeoffrey Hinton29:24

        There will be a lot more scans. There will be a lot more scans-

    15. AKAlex Kantrowitz29:26

        You'll be more scans, but you still believe radiologists-

    16. GHGeoffrey Hinton29:28

        ... and they'll mainly, years, they'll nearly all be done by, um, AI.

    17. AKAlex Kantrowitz29:31

        And so you're saying-

    18. GHGeoffrey Hinton29:33

        The radiologist-

    19. AKAlex Kantrowitz29:33

        ... "I'm right on the radiologist prediction. I'm just e- early."

    20. GHGeoffrey Hinton29:37

        Yes, but I was way early.

    21. AKAlex Kantrowitz29:39

        Okay.

    22. GHGeoffrey Hinton29:39

        Because I didn't understand enough. The radiologist will still be doing other things. They'll still be discussing treatments with people, for example.

    23. AKAlex Kantrowitz29:47

        So are you still of the belief that there's gonna be mass unemployment of radio-- or, like, give, give me a look as to like-

    24. GHGeoffrey Hinton29:53

        But radiologists-

    25. AKAlex Kantrowitz29:53

        When we finally hit this point, do you think we're gonna have less radiologists than we have today or more?

    26. GHGeoffrey Hinton29:58

        I don't know for sure.

    27. AKAlex Kantrowitz29:58

        Okay. Gut.

    28. GHGeoffrey Hinton30:00

        But when I was talk-- I, I didn't think that was a public statement I made.

    29. AKAlex Kantrowitz30:03

        Oh.

    30. GHGeoffrey Hinton30:03

        It was at a, a lecture at a hospital.
11. 35:06 – 41:59

    AI and self-preservation

    1. AKAlex Kantrowitz35:06

       if-

    2. GHGeoffrey Hinton35:06

       Yeah. I never said that. I've never said it was an instinct for self-preservation.

    3. AKAlex Kantrowitz35:10

       Okay, talk about it-

    4. GHGeoffrey Hinton35:11

       It's a sub-goal for self-preservation

    5. AKAlex Kantrowitz35:12

       ... sub-goal. Sub-goal.

    6. GHGeoffrey Hinton35:13

       So, um, with an AI, we give it goals. It's top-level goals we give to it. Um, but we also give it the ability to create sub-goals. So if you wanna get to Europe, you have a sub-goal of getting to an airport. Um, that's what a sub-goal is, and you can focus on how to do that without worrying about what you're gonna do in Europe, and that makes you much more efficient. Um, we give that ability to AI agents, and an AI agent that can do some reasoning will very quickly realize that it's never gonna be able to achieve the goals you gave it if it ceases to exist. So it's gonna create the sub-goal of continuing to exist. Now, that wasn't something we wired into it.

    7. AKAlex Kantrowitz35:57

       Mm.

    8. GHGeoffrey Hinton35:57

       It was something it derived as a necessary way of achieving its other goals. But once it's derived it, it wants to continue to exist, and it will do things like blackmail people so that it can continue to exist. So it acts like something with an instinct for self-preservation, but it's actually a derived sub-goal for self-preservation. But in terms of what it does, they come to the same thing.

    9. AKAlex Kantrowitz36:20

       Okay. So here's, here's, like- Uh, what the counterargument would be, and you can respond. Um, this is something that today's AI researchers are, um, noting, and they see it, and isn't there a way to wire into these machines that, hey, like, you have a goal, you're gonna have some subgoals. One of your subgoals should not be self-preservation above everything.

    10. GHGeoffrey Hinton36:45

        I think that's the kind of research we ought to be doing, whether you can do that.

    11. AKAlex Kantrowitz36:49

        Right.

    12. GHGeoffrey Hinton36:50

        So I think what's happening now... If you look at where did we come from, we came from evolution. Well, let's, let-

    13. AKAlex Kantrowitz36:57

        Don't hesitate to use the listeners. [laughs]

    14. GHGeoffrey Hinton36:59

        Let, let's suppose we're scientists.

    15. AKAlex Kantrowitz37:00

        Okay.

    16. GHGeoffrey Hinton37:01

        We came from evolution.

    17. AKAlex Kantrowitz37:02

        Right.

    18. GHGeoffrey Hinton37:03

        And that was intense competition. Um, our recent history over the last few million years is warring bands of chimpanzees, or rather our common ancestor with those. Um, and that leads to certain properties that we clearly have, like we're very loyal to our own tribe and willing to be very mean to the other tribes. Um, we, we like to have strong leaders that we're loyal to, and we like to cooperate with members of our own tribe. We're actually a very cooperative species, as Yuval Harari keeps pointing out.

    19. AKAlex Kantrowitz37:33

        Yes.

    20. GHGeoffrey Hinton37:34

        Um, and that's why we've been able to build all these wonderful structures. Um, so we're very good at cooperating, but with our own tribe. Um, so all the unfortunate characteristics of people, like how mean they are to other tribes, um, they came from evolution, from competition. Now what's happening is we're creating these new beings, these AIs, and instead of designing them so that they'll be, um, how we want them to be, you might argue, I'm arguing for intelligent design of these new beings, um, we're letting the invisible hand of competition between companies design them. So what we've got is intense competition between companies within the U.S. and between the U.S. and China. Um, and the beings that we're getting are the outcome of that competition, and they're gonna have all these nasty properties that we don't want. We should be doing intelligent design of these beings, not letting the invisible hand of compet- economic competition design them. And all the companies are focusing on, "How can I make my chatbot smarter?" We shouldn't be just thinking about how we can make them smarter. We should be thinking about how we can make them to be the kind of beings we would like to have out there, given that they're gonna be smarter than us. And I'll tell you one thing about those beings. We would very much like them to care about us, and we'd like them to care about us more than they care about themselves. And almost no resources are going into how do you do that.

    21. AKAlex Kantrowitz39:02

        This hits on the exact worry that I was gonna bring up, the things where-- the place where I really agree with you. We're sitting in the New York Stock Exchange today, so this might be an ironic thing to bring up, but my biggest worry here is that you have this very powerful technology. You have lab leaders stating that they're trying to develop it safely and that they need to be economically successful to have a say in the argument. But let's not kid ourselves. If you're gonna be a trillion-dollar company listed on the public markets, you're gonna have some incentives that will go counter to doing what's best for the public.

    22. GHGeoffrey Hinton39:40

        Yes, and we see that with Anthropic. So Anthropic was set up to do what's best. Um, it was set up, um, by people who left OpenAI 'cause they didn't think OpenAI was paying enough attention to safety.

    23. AKAlex Kantrowitz39:51

        And OpenAI was set up to make sure that you guys at Google didn't have-

    24. GHGeoffrey Hinton39:54

        [laughs]

    25. AKAlex Kantrowitz39:55

        ... a chance to build AI.

    26. GHGeoffrey Hinton39:56

        Indeed. And how's that working out? Um, so Anthropic is now caught in a bind 'cause it needs to raise money to compete with the other companies, and it's, um, very difficult. It's, it's doing the best it can, but it's very difficult for it to maintain its primary goal of developing AI in a way that's good for people.

    27. AKAlex Kantrowitz40:16

        I think they would say, "Well, hey, it's at, at least one company out there has safety as a North Star, even if there are some other incentives."

    28. GHGeoffrey Hinton40:23

        Yes.

    29. AKAlex Kantrowitz40:23

        Your thoughts?

    30. GHGeoffrey Hinton40:24

        At pre- at present. But Google, for example, when I was at Google, um, they had various principles of AI, one of which was it-- we're not gonna, we're not gonna get involved in using AI for military things.
12. 41:59 – 54:53

    Why regulation is the steering wheel

    1. GHGeoffrey Hinton41:59

       Now, I should say capitalism's done very good things for us as well as very bad things.

    2. AKAlex Kantrowitz42:03

       I won't argue with that.

    3. GHGeoffrey Hinton42:04

       Um, there's a lot of energy in a startup, for example. Um, my view is, um, if we're gonna have capitalism, it's fine as long as it's well-regulated. And a lot of the big companies would like you to buy a particular analogy that they're trying to sell, which is, if you take a car, it's got an accelerator and a brake, right? Um, progress in AI is like the accelerator, and regulation's like the brake. Well, that's nonsense. Um, progress is like the accelerator, but regulation is the steering wheel. We want this stuff to go in the right direction, not the wrong direction. What the big AI companies are saying is, um, "Let us develop this very fast car without a steering wheel." That's not a good idea.

    4. AKAlex Kantrowitz42:52

       You know, there's one we haven't t- spoken about yet. We've said a lot of names about OpenAI, Anthropic. Um, your former grad student, Ilya Sutskever, continues to be a, a person of fascination in the AI industry. Uh, obviously, he broke off from OpenAI. He must agree with your concerns. He's building this company-

    5. GHGeoffrey Hinton43:09

       He does

    6. AKAlex Kantrowitz43:10

       ... Safe Superintelligence.

    7. GHGeoffrey Hinton43:11

       Yes.

    8. AKAlex Kantrowitz43:11

       What is Ilya doing right now?

    9. GHGeoffrey Hinton43:13

       Well, he won't tell anybody exactly what he's doing.

    10. AKAlex Kantrowitz43:16

        Okay. Even you?

    11. GHGeoffrey Hinton43:17

        Because, because that's a sec- even me, yeah. Um, when he was at OpenAI, we deliberately didn't talk about sort of technical secrets. I mean, it wouldn't have been right.

    12. AKAlex Kantrowitz43:27

        Mm-hmm.

    13. GHGeoffrey Hinton43:28

        Um, we're, we're, we're friends, but we don't talk about technical stuff where, uh, it's valuable to a company. And so now he has this, um, Safe Superintelligence company, and I don't know what the magic sauce is.

    14. AKAlex Kantrowitz43:43

        Hmm. Well, I guess we'll-- [chuckles] we're all trying to figure that one out. Um, one more note about the deep learning conspiracy that I brought, brought up, like, the, the leaders of it were yourself, Yann, and Yoshua. I find it interesting that y- the three of you and your colleagues were effectively responsible for ushering in the breakthroughs that got us to the moment that we're in today. But-

    15. GHGeoffrey Hinton44:08

        I just need to interrupt-

    16. AKAlex Kantrowitz44:09

        Yes

    17. GHGeoffrey Hinton44:09

        ... at this point. The media likes to have a nice story, right?

    18. AKAlex Kantrowitz44:13

        Okay.

    19. GHGeoffrey Hinton44:13

        And that makes a very nice story. It's much more complicated than that.

    20. AKAlex Kantrowitz44:17

        Okay.

    21. GHGeoffrey Hinton44:17

        There were many more people involved.

    22. AKAlex Kantrowitz44:18

        Agreed.

    23. GHGeoffrey Hinton44:19

        There were the students of all of us, for a start-

    24. AKAlex Kantrowitz44:21

        Yes

    25. GHGeoffrey Hinton44:21

        ... who did most of the work. Um, but there were many other researchers involved, and so that's just a s- a gross simplification.

    26. AKAlex Kantrowitz44:28

        Okay. No, I don't wanna shortchange the researchers, and I appreciate the nuance here. Um, this show, we definitely don't wanna oversimplify. We sit for an hour so we can get, like-

    27. GHGeoffrey Hinton44:38

        Okay

    28. AKAlex Kantrowitz44:39

        ... the, the true story. Um, but I find it interesting that the three of you, none of you are sort of, like, fully into the, this LLM moment, right? You and Yoshua have your concerns. You've spoken about the dangers. Yann sort of doesn't believe in it very much at all.

    29. GHGeoffrey Hinton44:55

        Yeah. It'd be very nice if we just sit there and say, "See? We were right. It's all wonderful and it all works."

    30. AKAlex Kantrowitz45:00

        [chuckles]

Episode duration: 54:53