Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures

1. 0:00 – 11:41

   Measuring AGI

   1. DPDwarkesh Patel0:00

      Okay. Today, I have the pleasure of interviewing Shane Legg, who is a founder and the chief AGI scientist of Google DeepMind. Shane, welcome to the podcast.

   2. SLShane Legg0:12

      Thank you. It's a pleasure to be here.

   3. DPDwarkesh Patel0:14

      So first question, how do we measure progress towards AGI concretely? So we have these loss numbers, and we can see how the loss improves from one model to another, but it's just a number. How do we interpret this? How do we see w- how much progress we're actually making?

   4. SLShane Legg0:28

      That's a, that's a hard question (laughs) actually. Um, AGI, by its definition, is about generality. So it's not about doing a specific thing. It's much easier to measure performance when you have a very specific thing in mind, because you can construct a test around that. Well, maybe I should first of all explain, what do I mean by AGI? 'Cause there are a few different notions around. When I say AGI, I mean, um, a, a machine that can do the sorts of, uh, cognitive things that people can typically do, possibly more. But that's, to be an AGI, that's kind of the, the bar you need to meet. So if we want to test whether we're, we're meeting this threshold or we're getting close to the threshold, what we actually need then is, um, a lot of different kinds of measurements and tests of all the, spans the breadth of all the sorts of cognitive tasks that people can do. And then to have a sense of what is human performance, you know, on the, on these sorts of tasks, and that'll then allows us to sort of judge whether or not we're, we're there. It's difficult because you'll never have a complete set of everything that people can do, because it's, you know, such a large set. But I think that if you ever get to the point where you have a, have a, have a pretty good range of tests of all sorts of different things that people do, cognitive things that people can do, and you have an AI system which can meet human performance and all those things, and with some effort you can't actually come up with new examples of cognitive tasks where the machine is below human performance, then at that point, it's conceptually possible that there is something that the, um, the machine can't do that people can do. But if you can't find it with some effort, I think for all practical purposes, you now have an AGI.

   5. DPDwarkesh Patel2:11

      So, uh, let's get more concrete. Um, and, uh, bl- you know, we measure the performance of these large language models on MMLU or something, and maybe you can explain what all these different benchmarks are. But the ones we use right now that you might see in a paper, what, what are they missing? What aspect of human cognition do they not measure a- adequately?

   6. SLShane Legg2:31

      Ooh, yeah. Another hard question. (laughs)

   7. DPDwarkesh Patel2:33

      (laughs)

   8. SLShane Legg2:34

      These are, these are quite big, uh, areas. So they don't measure things like, uh, understanding streaming video, for example, because these are language models and people can do things like understanding streaming video. Um, they don't do things, like humans have what we call episodic memory, all right? So we have a working memory, which are things that have happened quite recently, and then we have sort of a cortical memory. So these are things that are sort of been, you know, in, in our cortex that have been. There's also a system in between, which is episodic memory, which is the hippocampus. And so this is about learning, um, specific things very, very rapidly. So some of the things I say to you today, if you remember them tomorrow, that will be your, your hip- your episodic memory hippocampus.

   9. DPDwarkesh Patel3:16

      Yeah.

   10. SLShane Legg3:16

       Our models don't really have that kind of thing, and we don't really test for that kind of thing. We just sort of try to make the context windows, which is, I think more like a working memory, longer and longer to sort of compensate for this. But yeah, we don't, we don't really test for that kind of a thing. Um, so there is, there is all sorts of bits and pieces, but you know, it is, it is a difficult question because you really need to, as I said, int- intelligence, the generality of human intelligence is, is very, very broad. So you really have to start going into the weeds of trying to find, you know, these specific types of things that are missing from existing benchmarks or different categories of benchmarks that, you know, um, don't currently exist or something. Yeah.

   11. DPDwarkesh Patel3:53

       Mm-hmm. Uh, the, the thing you're referring to with episodic memory, would it be fair to call that sample efficiency or is that a different, uh...

   12. SLShane Legg4:01

       Uh, it's, it's very much related to sample efficiency. It's, it's one of the things that enables humans to be very sample efficient.

   13. DPDwarkesh Patel4:06

       Right.

   14. SLShane Legg4:07

       Um, large language models have a certain kind of sample efficiency because when something's in their context window, they can then, that, that sort of biases the distribution to, to behave in a different way. And so that's a very rapid kind of learning. So there are multiple kinds of learning and, uh, the existing systems have some of them, but not others. So it's, it's a little bit complicated.

   15. DPDwarkesh Patel4:30

       So, uh, this kind of memory, or, uh, we call it sample efficiency, whatever, uh, is, is it a fatal flaw of these deep learning models, that it just takes trillions of tokens, m- far more, many orders of magnitude more than a human will see throughout their lifetime? Or is this something that just solved over time?

   16. SLShane Legg4:46

       So it, the models can learn things immediately when it's in a context window, and then they have this sort of, this longer process of when you actually train the base model and so on. Um, and that's, they're learning over trillions of tokens, but they sort of miss something in the middle.

   17. DPDwarkesh Patel5:00

       Right.

   18. SLShane Legg5:00

       Right? That's sort of the, what I'm getting at here. Um, I don't think it's a fundamental limitation. Um, I think what's happened with, uh, large language models is something fundamental has changed. We know how to build models now that have some degree of, I would say, understanding of, of what's going on. And that did not exist in the past. And because we've got a scalable way to do this now, that unlocks lots and lots of, lots of new things. Now we can then look at things which are missing, such as this sort of episodic memory type thing, and we can then start to imagine ways to address that. So my feeling is that the, there are...... kind of relatively clear paths forwards now to address most of the shortcomings we see in existing models, whether it's about delusions, factuality, the, the type of memory and learning that they have, or understanding video, or all sorts of things like this. So I, I'm not actually... I don't see there are big blockers here. I don't see big walls in front of us. I just see there's more research and work, and these things wi- will improve and, and probably be adequately solved.

   19. DPDwarkesh Patel6:08

       Mm. But going back to the original question of how do you measure when a human-level AI is arrived or beyond it, uh, uh, you... As you mentioned, there's these other sorts of benchmarks you can use and other sorts of traits. But concretely, if there is... Wh- what would it have to do for you to be like, "Okay, we've reached human level"? Would it have to beat Minecraft from start to finish? Would it have to get 100% on MMLU? What would it have to do?

   20. SLShane Legg6:32

       There is no one thing that would do it because I think that's the nature of it. It's about general intelligence, so it'd have to make sure it could do lots and lots of different things and it didn't have a gap. We already have systems that can do very impressive categories of things to human level or even beyond.

   21. DPDwarkesh Patel6:47

       Yeah.

   22. SLShane Legg6:48

       So I would want a, a whole suite of tests that I felt was very comprehensive. And then furthermore, when people come and say, "Okay, so it's passing our big suite of tests," let's try to find examples. Let's be, take an adversarial approach to this. Let's deliberately try to find examples where people can clearly typically do this but the machine fails. And when those people cannot succeed, I'll go, "Okay, we're probably there."

   23. DPDwarkesh Patel7:16

       A lot of your earlier research, at least the one that I, Zach, I find, emphasized the... that AI should be able to manipulate and succeed in a variety of open-ended environments.

   24. SLShane Legg7:27

       Yes.

   25. DPDwarkesh Patel7:28

       It kinda sounds like a video game almost. Is that where your head is still at now, or do you think about it differently?

   26. SLShane Legg7:32

       Yeah. It's evolved a bit. When, when I did my thesis work around, uh, universal intelligence and so on, um, I was trying to come up with a sort of e- extremely universal, general, mathematically clean framework for defining and measuring intelligence. Um, and I, I think there were aspects of that that were successful. I think it... In my own mind, it clarified, um, the nature of intelligence being able... as being able to perform well in lots of different domains and different tasks and so on. It's about that sort of capability of performance and, and, and the breadth of performance. Um, so I, I found that was, that was quite helpful, enlightening. There was always the issue of the reference machine, because you... In, in the, um, in the framework, you have a, a weighting of things according to the complexity. It's like an Occam's razor type of thing, where you weight, um, tasks, environments which are simpler h- more highly in this sort of... 'Cause you've got, you've got an infinite... Uh, it's a, it's a, it's a countable space of, of different, uh, computable environments, of semi-computable environments. Um, and that Kolmogorov complexity measure has something built into it which is called a reference machine, and that's a free parameter. So that means that the, the intelligence measure has a free parameter in it, and as you change that free parameter, it changes the weighting and the distribution over the space of all the different tasks and environments. So this is sort of an unresolved part of the whole problem. So what reference machine should we ideally use?

   27. DPDwarkesh Patel9:17

       Mm-hmm.

   28. SLShane Legg9:18

       There isn't really a... There's no universal, like, one specific reference machine. People will usually put a Universal Turing machine in there, but there are many kinds of Universal Turing machines, so you have to put in, sorry, a Universal Turing machine, but there are many different ones. So I think given that it's a free parameter, I think the most natural thing to do is say, "Okay, let's think about what's meaningful to us in terms of intelligence." Uh, I think human intelligence is meaningful to us and the environment that we live in.

   29. DPDwarkesh Patel9:49

       Mm-hmm.

   30. SLShane Legg9:49

       We, we know what human intelligence is. We are human too. We interact with other people who have human intelligence. We know that human intelligence is possible, obviously, because, you know, it exists in the world. We know that human intelligence is very, very powerful because it's affected the world profoundly and in countless ways. Um, and we know if human-level intelligence was achieved, that would be economically transformative because the types of cognitive tasks people do in the economy would... could be done by machines then. And it would be, uh, philosophically important because this is sort of a... how we often think about, you know, intelligence. And I think historically would be a key point. So I think that human intelligence is actually quite... in, in a human-like environment is quite a natural sort of reference point.
2. 11:41 – 16:26

   Do we need new architectures?

   1. SLShane Legg11:41

   2. DPDwarkesh Patel11:41

      Uh, so b- before we move on, I, I do want to ask, uh, on the original point you made about these machines or these LLMs need, um, episodic memory-Uh, you said that these are problems that we can solve, these are not fundamental imp- impediments. But are- when you say that, do you think they will just be solved by skill, or do each of these need a fine-grain specific solution that is architectural in nature?

   3. SLShane Legg12:05

      I- I- I think it'll be architectural in nature because the- well, the current architectures, they- they- they don't really have what you need to do this. They basically have a context window, which is very, very fluid, of course, and they have the weights, which things get baked into very slowly. So to my mind, that feels like working memory, which is like the a- the activations, uh, in your brain, and then the weights, the synapses and so on in your cortex. Now, the- the brain separates these things out. It has a separate mechanism for lear- rapidly learning specif- specific information-

   4. DPDwarkesh Patel12:39

      Mm-hmm.

   5. SLShane Legg12:39

      ... because that's a different type of optimization problem compared to slowly learning deep generalities, all right? There's sort of-

   6. DPDwarkesh Patel12:47

      Right, yeah, yeah.

   7. SLShane Legg12:47

      There's a- there's a- there's a tension between the two. But you want to be able to do both. You want to be able to, I don't know, hear someone's name and remember it the next day, and you also want to be able to integrate information over a lifetime, so you start to see deeper patterns in the world. These are- these are quite different- different, um, learn- uh, optimization targets, different, you know, processes. But, uh, y- a comprehensive system should be able to do both. And so I think it's- it's conceivable you could build one system does both, but you can see because they're quite different things that it makes sense for them to be different, and I think that's why the brain does it separately.

   8. DPDwarkesh Patel13:21

      Mm. I- I'm curious about how concretely you think that would be achieved and I'm s- specifically curious, um, I- I guess you can answer this as part of the answer, you know, DeepMind has been working on these domain-specific reinforcement learning type setups, AlphaFold, AlphaCode, and so on.

   9. SLShane Legg13:39

      Mm-hmm.

   10. DPDwarkesh Patel13:39

       How does that fit into what you see as a path to AGI? Have the- these just been orthogonal domain-specific models or do they feed into the eventual AGI?

   11. SLShane Legg13:49

       Uh, things like AlphaFold are not really feeding into AGI. Um, you know, we may learn things in the process-

   12. DPDwarkesh Patel13:57

       Right.

   13. SLShane Legg13:58

       ... uh, that- that may end up being relevant, but I- I don't see them as being- likely being on the path to AGI.

   14. DPDwarkesh Patel14:04

       Yeah.

   15. SLShane Legg14:05

       Um, but yeah, we're- you know, we're a big group.

   16. DPDwarkesh Patel14:07

       Yeah.

   17. SLShane Legg14:07

       We've got hundreds and hundreds and hundreds of PhDs working on lots of different projects, so, you know, when we find, um, you know- well, we see like opportunities to- to do something significant like AlphaFold, we'll- we'll go and do it.

   18. DPDwarkesh Patel14:19

       Right.

   19. SLShane Legg14:19

       It's not like we- we only do AGI type work. We- we- we work on fusion reactors and, um, you know, uh, various things in, uh, sustainability, energy. We've got people looking at, um, you know, satellite images of- um, of, uh, deforestation. We have people looking at, uh, weather forecasting. We got tons of people working on lots of things. (laughs)

   20. DPDwarkesh Patel14:43

       On the point you made earlier about where the- the reference class or the reference machine is human intelligence, it's interesting because in your 2008 thesis, one of the things you mentioned almost as a side note is how- well, how would you measure intelligence? And you said, "Well, you could do a compression test and you could see if it fills in words in a sample of text and that can measure intelligence." And funnily enough, that's basically how LLMs are trained. At the time, did it stick out to you as a especially fruitful thing to train for?

   21. SLShane Legg15:11

       Well, yeah. I mean, in a sense what's happened is actually very aligned with, um, what I write about in my thesis-

   22. DPDwarkesh Patel15:19

       Mm.

   23. SLShane Legg15:19

       ... which are the ideas from Marcus Hutter, um, with AIC, where, uh, you take Solomonic induction, which is this incomputable but sort of theoretically very elegant and extremely, uh, sample efficient, uh, prediction system. Um, and then once you have that, you can build a- a general agent on top of it by basically adding, um, search and, uh, reinforcement signal. That's what you do with AIC. Um, but what that sort of tells you is that if you have a fantastically good sequence predictor, some approximation of Solomonic induction, then going from that to a very powerful, very general AI sys- AGI system is- is just sort of an- an- another step, you know, it's- it's you've actually solved a lot of the problem already.

   24. DPDwarkesh Patel16:09

       Mm-hmm.

   25. SLShane Legg16:09

       And I think that's what we're seeing today actually, that these incredibly powerful foundation models are incredibly good sequence predictors, they're compressing the world based on all this data, and then you could- you will be able to extend these in different ways and build very, very powerful agents out of them.

   26. DPDwarkesh Patel16:25

       Mm. All right,

3. 16:26 – 19:19

   Is search needed for creativity?

   1. DPDwarkesh Patel16:26

      let me ask you more about that. So Richard Sutton's, uh, uh, Bitter Lesson essay says that there's two things you can scale, um, search and learning. And I guess you could say that LLMs are about the learning aspect.

   2. SLShane Legg16:37

      Yeah.

   3. DPDwarkesh Patel16:37

      Um, the- the search stuff, which you've worked on throughout your career, where you have an agent that is ac- uh, you know, interacting with this environment and is that- uh, is that's a- the direction that needs to be explored again or- or is that something that needs to be added to LLMs where they can actually interact with their data or the world or in some way?

   4. SLShane Legg16:53

      Yeah. Um, I- I- I think that's on the right track. I think there is a l- these foundation models are world models of a kind.

   5. DPDwarkesh Patel17:03

      Mm-hmm.

   6. SLShane Legg17:03

      And to do really creative, um, problem solving, you need to start searching. So if I think about something like AlphaGo and the move 37, the famous move 37, where did that come from? Did that come from all its data that it's seen of human games or something like that? No, it didn't. It came from it identifying a move as being quite unlikely but, you know, plausible and then via process of search coming to understand that the- that was actually a very, very good move. So you need to s- you- to get real creativity, you need to search through spaces of possibilities and find these sort of hidden gems, that's what creativity is. I think current language models, they don't really do that kind of a thing.

   7. DPDwarkesh Patel17:50

      Mm-hmm.

   8. SLShane Legg17:50

      They really are mimicking the data, they are mimicking all the human ingenuity and everything which they have seen-... from all this data that's coming from the internet that's originally derived from humans. If you want a system that can go be, re- truly beyond that and not just generalize in novel ways, so it can, you know, these models can blend things. They can do, you know, Harry Potter in the style of a Kanye West rap or something-

   9. DPDwarkesh Patel18:15

      Yeah.

   10. SLShane Legg18:15

       ... even though it's never happened. They can blend things together-

   11. DPDwarkesh Patel18:18

       Right.

   12. SLShane Legg18:18

       ... but to do something as truly creative that ha- that is not just a blending of existing things, that requires searching through a space of possibilities and finding these hidden gems that, that are, that are sort of e- they're hidden away in there somewhere and that requires search. So I don't think we'll see systems that truly step beyond their training data until we have powerful search in the process.

   13. DPDwarkesh Patel18:43

       So there are rumors that Google DeepMind is training newer models, and you don't have to comment on those specifically, but when you do that, uh, if it's a, if it's the case that search or something like that is required to go to the next level, are you training in a completely different way than, say, GPT-4 or other transformers are trained?

   14. SLShane Legg19:01

       I can't say much about how we're training. Um, I think it's fair to say we're doing the sorts of scaling and training roughly that you see many people in the, in the field doing. Um, but we have, you know, our own take on it and our own different tricks and techniques.

4. 19:19 – 29:58

   Superhuman alignment

   1. DPDwarkesh Patel19:19

      Okay. Maybe we'll come back to it if we get, uh, another answer on that. But, uh, let's talk about alignment briefly. So what will it take to align human-level and superhuman, um, AIs? And you know, the, w- it's interesting because the sorts of reinforcement learning and self-play kinds of setups that are popular now, like Constitution AI or RLHF, DeepMind obviously has expertise in it for, for more decades longer. So I'm curious what you think of the current landscape and how DeepMind, uh, pursues that problem of safety towards human-level models.

   2. SLShane Legg19:50

      So do you want to know about what we're currently doing or do you want me to have a stab at what I think we, that i- needs to be done? (laughs)

   3. DPDwarkesh Patel19:55

      Needs to be done.

   4. SLShane Legg19:56

      Needs to be done. So I mean, what, in terms of what we're, we're currently doing, we're doing lots of things. We're doing interpretability. We're doing, uh, process supervision. We're doing red teaming. We're doing evaluation for dangerous capabilities. We're doing work on institutions and governance and, you know, tons of stuff, right? There's lots of different things. Anyway, what do I think needs to be done? (laughs)

   5. DPDwarkesh Patel20:17

      Yes.

   6. SLShane Legg20:17

      So I think, uh, I, I think that powerful machine learning, powerful AGI is coming in some, sometime.

   7. DPDwarkesh Patel20:25

      Right.

   8. SLShane Legg20:26

      And if the system is really capable, really intelligent, really powerful, trying to somehow contain it or limit it is probably not a winning strategy because these systems ultimately will be very, very capable.

   9. DPDwarkesh Patel20:39

      Mm-hmm.

   10. SLShane Legg20:40

       So what you have to do is you have to align it.

   11. DPDwarkesh Patel20:42

       Mm-hmm.

   12. SLShane Legg20:42

       You have to get it so it's fundamentally a highly ethical, value-aligned system from the get-go, right? How do you do that? Well, I, I, I have a... Maybe this is slightly naive, but this is, this is my take on it. How do people do it, right? If you have a really difficult ethical decision in front of you, what do you do, right? Well, you don't just do the first thing that comes to mind, right? Because, you know, there could be a lot of emotions involved and other things, right? It's a difficult problem. So what you have to do is you have to calm yourself down, you gotta sit down, and you gotta think about it. You gotta think, "Well, okay. What, what could I do? I could do this, I could do this, I could do this. If I do each of these things, what will happen," right? And then you have to think about... So that requires a model of the world. And then you have to think about... Ethically, how do I view each of these different actions and the possibilities and wha- what may happen from it, right? What is the right thing to do? And as you think about all the different possibilities and w- your actions and what can follow from them and, and, and how it aligns with your values and your ethics, you can then come to some conclusion of what is really, you know, the best choice that you should be making if you want to be, you know, really ethical about this. I think AI systems need to essentially do the same thing. So when you sample from a foundation model at the moment, it's like it's blurting out the first thing. It's like System 1, if you like, from psychology from Kahneman, right? Um, that's, that's not good enough. And if we do RLHF or, um, what is it called? I can't remember. Anyways, the AI version without the hu- the human feedback. RAIF? Is that what it is? Oh, gosh, I'm confusing myself. Anyway, Constitutional AI tries to do this sort of thing. You're trying to fix the underlying System 1 in a sense, right? And that can shift the distribution and that can be very helpful but it's a very high-dimensional distribution and you're sort of poking it in a whole lot of points. And so it's not likely to be a very robust solution, right? It's like trying to train yourself out of a bad habit. You know, you can sort of do it eventually. What you need to do is you need to have a System 2. You need a system to not just sample from the model. You need a system to go, "Okay, I'm gonna reason this through. I'm gonna do, uh, step by step reasoning. What are the options in front of me? I'm gonna use my world model now and I'm gonna use a good world model to understand what's likely to happen from each of these options and then reason about each of these from an ethical perspective." So you need a system which has a, a, a deep understanding of the world, has a good world model. It has a good understanding of people, it has a good understanding of ethics, and it has robust and very reliable reasoning. And then you set it up in such a way that it applies this reasoning and this understanding of ethics to analyze the different options which are in front of it and then execute on which is the most eth- ethical, um, way forwards.

   13. DPDwarkesh Patel23:52

       But I, I think when, uh, a lot of people think about the fundamental alignment problem, the worry is not that it's not going to have a world model necessary to understand its actions-... I'm sorry, to understand that the effects of its, uh, actions. I guess it's one worry, but n- not the main worry. The, the main worry is that the effects it cares about are not the ones we will care about. And so even if you improve its systems, your thinking, and do better planning, the fundamental problem of we have this really nuanced values about what we want-

   14. SLShane Legg24:21

       Mm-hmm.

   15. DPDwarkesh Patel24:21

       ... how do we communicate those values and make sure they're reinforced in the, uh, AI?

   16. SLShane Legg24:26

       It needs not just a good model of the world, but it needs to have g- really good understanding of ethics. And we need to communicate to the system what ethics and values it should be following.

   17. DPDwarkesh Patel24:36

       And how do we do that in a way that's, uh, we can be confident that a human level or eventually a superhuman level model will preserve those values or learn them in the first place?

   18. SLShane Legg24:45

       Well, it should preserve them. Because if it's making all its decisions based on a good understanding of ethics and, and values, and it's consistent in doing this, it shouldn't take actions which undermine that. That would be, that would be inconsistent.

   19. DPDwarkesh Patel24:58

       Right. So then how do we get to the point where it's learned them in the first place?

   20. SLShane Legg25:01

       Yeah. That's the challenge.

   21. DPDwarkesh Patel25:03

       Yeah.

   22. SLShane Legg25:03

       We need to have systems... I, the way I think about it is this. To have a profoundly ethical AI system, it also has to be very, very capable. It had, needs a really good world model, a really good understanding of ethics, and it needs really good reasoning. Because if you don't have any of those things, how can you possibly be consistently profoundly ethical? You can't. So we actually need better reasoning, better understanding of the world, and better e- better understanding of ethics in our systems.

   23. DPDwarkesh Patel25:33

       Right. So it seems to me the former two would just come along for the ride as these models get more powerful.

   24. SLShane Legg25:38

       Yeah. So that's a nice property because it's actually a capabilities thing-

   25. DPDwarkesh Patel25:41

       Right.

   26. SLShane Legg25:41

       ... to some extent.

   27. DPDwarkesh Patel25:42

       But then if the third one is a bottleneck or if the third one is a thing that d- doesn't come along with the AI itself, w- what, what is the actual technique to make sure that that happens?

   28. SLShane Legg25:51

       The third one, sorry, the third one-

   29. DPDwarkesh Patel25:52

       The, the, the ethical model. What, what do humans value?

   30. SLShane Legg25:54

       We... Well, we've got t- we've got, we've got a couple problems. First of all, we need to decide w- we, we should train the system on ethics generally. I mean, there's a lot of, you know, lectures and papers and books and all sorts of things.
5. 29:58 – 34:03

   Impact of Deepmind on safety vs capabilities

   1. DPDwarkesh Patel29:58

      Um, so, you know, it's... uh, you have... it's, it's interesting 'cause you have these, uh, blog posts that you wrote when you started DeepMind-

   2. SLShane Legg30:04

      Yeah.

   3. DPDwarkesh Patel30:04

      ... um, you know, back in 2008, uh, where you talk about... um, it, the, the motivation was, uh, to accelerate safety.

   4. SLShane Legg30:11

      Mm-hmm.

   5. DPDwarkesh Patel30:12

      O- o- on net, what do you think the impact of DeepMind has been on safety versus capabilities?

   6. SLShane Legg30:17

      Oh, interesting. I don't know. It's hard to d- hard to judge actually. I- you know, back in the... I, I've been worried about AGI safety for a long time, bef- well before DeepMind. Um, but it was, it was always really hard to hire people actually, particularly in the early days, to work on AGI safety. Um, u- thinking back on 2000 like 13 or so, I think we had the first hire and he only agreed to do it part-time (laughs) because he didn't want to, you know, drop all the capabilities work because, you know, the impact it would have on his career and stuff. And this was someone who'd already previously been pud- pu- publishing in AGI safety. So yeah, I don't know. It's hard to, hard to know what is the counterfactual if we, if we weren't, weren't there doing it. Um, I think, you know, we have been... We've been a group that's been, um, you know, talked about this openly. I've, I've, I've talked about this on many occasions, the importance of it. Um, we've been, you know, hiring people to, to work on these topics. Um, you know, I know a lot of other people in the area and I've talked to them m- over many, many years.

   7. DPDwarkesh Patel31:34

      Yeah.

   8. SLShane Legg31:34

      I've known Dario since 2005 or something around there.

   9. DPDwarkesh Patel31:38

      Yeah.

   10. SLShane Legg31:38

       You know, we've talked on and off about AGI safety and so on. So I don't know. The, the impact that DeepMind has had, you know, we... I guess we were the first... I'd say the first AGI company, and as the first AGI company, we, we, you know, we always had an AGI safety group. Um, we al- we've been publishing papers in this for many years.

   11. DPDwarkesh Patel31:59

       Mm-hmm.

   12. SLShane Legg32:00

       I think that's lent some credibility to the area when people see, "Oh, here's, uh, AGI." I mean, AGI was a, you know... (laughs) It was a fringe term not that long ago.

   13. DPDwarkesh Patel32:08

       Yeah.

   14. SLShane Legg32:08

       And this person's doing AGI safety, and they're like, "Well, they're at DeepMind?"

   15. DPDwarkesh Patel32:11

       Yeah.

   16. SLShane Legg32:12

       "Oh, okay." Uh, I th- I, I hope that sort of, you know, creates some space for people.

   17. DPDwarkesh Patel32:17

       And w- w- where do you think A- AI progress itself would've been without DeepMind? And this is not just a point that people make about DeepMind. I think this is a general point people make about OpenAI and Anthropic as well, that these people went into the business to accelerate safety, and sort of the net effect might've been to accelerate capabilities far more.

   18. SLShane Legg32:33

       Right. Right, right, right. I think we have accelerated capabilities, but again, the counterfactuals are quite, quite difficult. I mean, we, we didn't do ImageNet, for example.

   19. DPDwarkesh Patel32:43

       Right.

   20. SLShane Legg32:43

       And ImageNet, I think, was very influential in, in attracting investment to the field.

   21. DPDwarkesh Patel32:48

       Yeah.

   22. SLShane Legg32:49

       Um, we did do AlphaGo, um, and that changed some people's minds. Um, but, you know, the, the community is a lot bigger than just DeepMind. I mean, we, we have... Well, uh, not so much now, but n- uh, because there are a number of other, you know, players with significant resources, but if you went back more than five years in the future, we were able to do, um, bigger projects with bigger teams and take on more ambitious things than, than a lot of the smaller academic groups, right?

   23. DPDwarkesh Patel33:20

       Mm-hmm.

   24. SLShane Legg33:20

       And so the sort of nature of the type of work we could do is a bit different.

   25. DPDwarkesh Patel33:24

       Mm-hmm.

   26. SLShane Legg33:25

       Um, and that, I think the... that affected the dynamics in some ways, but, you know, the, the, the community is much, much bigger than, say, DeepMind. So maybe we've sped things up a bit, but I think a lot of these things would have happened before too long anyway. I think, I think these off- often good ideas are kind of in the air, and, you know, as a, as a researcher, you know, when sometimes you publish something or you're about to publish something, you see somebody else who's got a very similar idea coming out with some good results. Um, I think often it's the time is right, right for things. So, you know, it's... I find it very hard to reason about the counterfactuals there.

   27. DPDwarkesh Patel34:02

       Mm-hmm.

6. 34:03 – 41:24

   Timelines

   1. DPDwarkesh Patel34:03

      Speaking of the early years, it's really interesting that in, uh, 2009 you had a blog post-

   2. SLShane Legg34:07

      Yeah.

   3. DPDwarkesh Patel34:07

      ... where you say, "My modal expectation of when we get human level AI is 2025, expected value is 2028." And this is before deep learning, this is b- when nobody's talking about AI. And it turns out, like if you... if the trends continue, this, this is not an unreasonable prediction. This was-

   4. SLShane Legg34:24

      Yeah.

   5. DPDwarkesh Patel34:24

      Uh, how did you... I mean, before all these trends came into effect, how did you have that accurate an estimate?

   6. SLShane Legg34:29

      Well, first I'd say it's not before deep learning.

   7. DPDwarkesh Patel34:31

      Oh.

   8. SLShane Legg34:32

      Um, deep learning was getting started around 2008.

   9. DPDwarkesh Patel34:36

      Oh, sorry. I meant to say before ImageNet.

   10. SLShane Legg34:38

       Before ImageNet. That was 2012.

   11. DPDwarkesh Patel34:40

       Yeah.

   12. SLShane Legg34:41

       Yeah. Um, so, well, I first formed those beliefs in about 2001 after reading Ray Kurzweil's The Age of Spiritual Machines, and I, I came to the conclusion he was, he was... There was two really important points that... In, in, in his book that I, I came to believe as true. One is that I, uh... computational power would grow exponentially for at least a few decades, and that the quantity of data in the world would grow exponentially for a few decades. And when you have exponentially increasing quantities of computation and data, then the value of highly scalable algorithms gets higher and higher. So then there's a lot of incentive to make a more scalable algorithm to harness all this compute and data.

   13. DPDwarkesh Patel35:31

       Mm-hmm.

   14. SLShane Legg35:31

       And so I thought it would be very likely that we'll start to discover scalable algorithms to do this. And then there's a positive feedback between all these things, because if your algorithm gets better at harnessing compute and data, then the value of the data and the compute goes up because it can be more effectively used, and so that drives more investment into these areas. If your compute performance goes up, then the value of the data goes up because you can utilize more data. So there are positive feedback loops between all these things. So that was, that was the first thing. And then the second thing was just looking at the trends. If the scalable algorithms were to, were to be discovered, then during the 2020s, it should be possible to start training models on significantly more data than a human would experience in a lifetime.... and I figure that that would be a time where, where big things would start to happen, and that would eventually unlock AGI.

   15. DPDwarkesh Patel36:26

       Mm.

   16. SLShane Legg36:26

       So that was, that was my reasoning process and I think we're now at that first part. I think we can start training models now, where the scale of the data is beyond what a human can experience in a lifetime. So I think this is the first unlocking step. And so, yeah, I think there's a 50% chance that I'm saying about 2028. Now, it's just a 50% chance. I mean, I'm, I'm sure what's gonna happen is we're gonna get to, you know, 2029: "Oh, Shane, you were wrong." It's like, come on, it's a 50% chance. (laughs) So yeah. I, I, I think it's, it's entirely plausible, yeah, it's a 50% chance it could happen by 2028.

   17. DPDwarkesh Patel37:01

       Mm.

   18. SLShane Legg37:02

       Um, but I'm not gonna be surprised if it doesn't happen by then. May- maybe, you know, the, you, you often hit, um, unexpected problems in, in research in sciences and sometimes things take longer than you expect.

   19. DPDwarkesh Patel37:13

       I- if there was a problem that caused it, if we're in 2029 and it hasn't happened yet-

   20. SLShane Legg37:17

       Yeah.

   21. DPDwarkesh Patel37:17

       ... looking back, what would be the most likely reason that would be the case?

   22. SLShane Legg37:22

       Whew. I don't know.

   23. DPDwarkesh Patel37:26

       (laughs)

   24. SLShane Legg37:28

       I don't know. I, at the moment, it looks to me like all the problems are likely solvable with a number of years of research. That, that's my current sense.

   25. DPDwarkesh Patel37:38

       And what does the time from here to 2028 look like, if the 2028 ends up being the year? Is it, is it just we have trillions of dollars of economic impact in the meantime, and would-

   26. SLShane Legg37:49

       Um...

   27. DPDwarkesh Patel37:49

       ... the world gets crazier? What happens?

   28. SLShane Legg37:51

       I think what you'll see is, um, the existing models maturing. Um, they'll be less delusional, much more factual. They'll be more up to date on what's currently going on when they answer questions. Um, they'll become multimodal, much more than they currently are, um, and this will just make them much more useful. So I think probably what we'll see more than anything is just, um, loads of great applications, um, for the, for the coming years.

   29. DPDwarkesh Patel38:21

       Mm.

   30. SLShane Legg38:21

       I think that'll be the, there'll, there can be some misuse cases as well. I'm sure somebody will come up with, you know, some- something to do with these, with these models that is, uh, quite unhelpful. But my expectation for the coming years is mostly a positive one. We'll see all kinds of really impressive, really amazing applications, um, for the, for the coming years. Yeah.
7. 41:24 – 44:18

   Multimodality

   1. SLShane Legg41:24

   2. DPDwarkesh Patel41:25

      Mm. So final question is this. You've been in this field for over a decade, l- much longer than many others, um, and you've seen these different landmarks, uh, ImageNet, transformers. What do you think the next landmark will look like?

   3. SLShane Legg41:41

      I think the next landmark that people will rem- will think back to and remember is going much more fully mul- multimodal, I think.

   4. DPDwarkesh Patel41:52

      Mm-hmm.

   5. SLShane Legg41:53

      Because I think that will, that'll open up the, the sort of understanding that you see in language models into a much larger space of possibilities.

   6. DPDwarkesh Patel42:02

      Mm.

   7. SLShane Legg42:02

      And when people think back, they'll think about, "Oh, those old-fashioned models. They, they just did like chat, they just did texts." You know? It was, it just felt like a very narrow thing. Whereas now, they, you know, they understand when you talk to them and they, they understand images and pictures and video and, and you can show them things or things like that and they'll, they will have much more understanding of what's going on. And it'll feel like the system's kinda opened up into the world in a, in, in a, in a much more powerful way.

   8. DPDwarkesh Patel42:29

      Do you mind if I ask a follow-up on that? So, uh, ChatGPT just released their multimodal feature and then you, in DeepMind, you had the GATO paper where, you know, you can, you have this one model, you can images, even actions, the video games, whatever you can throw in there. Um, and so far, it doesn't seem to have been... It hasn't percolated as much as even like ChatGPT initially from GPT-3 or something. What explains that? Is it just that people haven't learned to use mut- multimodality? They're not powerful enough yet?

   9. SLShane Legg42:54

      Uh, I think it's early days.

   10. DPDwarkesh Patel42:55

       Mm.

   11. SLShane Legg42:57

       Um, I think there's, you can see promise there understanding images and things, uh, more and more. But I think it's, yeah, it's early days in this transition, uh, is when you start really digesting a lot of video and other things like that, that the systems will start having a much more grounded understanding of the world and all kinds of other aspects.

   12. DPDwarkesh Patel43:15

       Mm-hmm.

   13. SLShane Legg43:15

       And then when that works well, that will open up naturally lots and lots of new, new applications and all sorts of new possibilities 'cause you're not confined to text chat anymore.

   14. DPDwarkesh Patel43:25

       The new avenues of training data as well, right?

   15. SLShane Legg43:28

       Yeah, new training data and new in all kinds of different applications that aren't just purely textual anymore. Um, and, you know, what are those applications? Well, probably a lot of them we can't even imagine at the moment because there are just so many, so many possibilities once you can start dealing with all sorts of different modalities in a consistent way.

   16. DPDwarkesh Patel43:46

       Awesome. Shane, I think that's an excellent place to leave it off.

   17. SLShane Legg43:48

       All right.

   18. DPDwarkesh Patel43:49

       Thank you so much for coming on the podcast.

   19. SLShane Legg43:50

       Thank you.

   20. DPDwarkesh Patel43:52

       Hey, everybody. I hope we enjoyed that episode. As always, the most helpful thing you can do is to share the podcast. Send it to people you think might enjoy it, put it in Twitter, your group chats, et cetera. Just blitz the world. Appreciate your listening. I'll see you next time. Cheers. (outro music)

Episode duration: 44:18