Demis Hassabis: Agents, AGI & The Next Big Scientific Breakthrough

1. 0:00 – 0:46

   Intro

   1. DHDemis Hassabis0:00

      Continual learning, long-term reasoning, uh, some aspects of memory, these are still unsolved. I think all of these are gonna be required for AGI. Depending on what your AGI timeline is, you know, mine's like 2030 or something like this, then if you start off on a deep tech journey n- today, you have to just consider AGI appearing in the middle of that journey. It's not bad necessarily, but you have to take that into account. You have to have an active system, uh, that can actively solve problems for you to get to AGI. So agents are that path, and I think we're just getting going.

   2. GTGarry Tan0:39

      Demis Hassabis has had one of the most unusual careers in tech.

2. 0:46 – 1:48

   Demis Hassabis: From Chess Prodigy to DeepMind

   1. GTGarry Tan0:46

      He was a chess prodigy as a kid, then designed his first hit video game, Theme Park, at 17. He then went back to school, got a PhD in cognitive neuroscience, published foundational work on how memory and imagination work in the brain, and then in 2010, co-founded DeepMind with one mission: solve intelligence, and I think they've done it. Uh, [laughs] since then-

   2. DHDemis Hassabis1:12

      [laughs]

   3. GTGarry Tan1:12

      ... uh, his lab has gone on to do things most people thought were decades away. AlphaGo beat a world champion at Go. AlphaFold cracked protein structure prediction, a 50-year grand challenge in biology, and they gave it away for free to every scientist on Earth. That work won him the Nobel Prize in Chemistry last year. Today, Demis leads Google DeepMind, where he's building Gemini and pushing toward the same goal he set when he was a teenager, artificial general intelligence. Please welcome Demis Hassabis.

3. 1:48 – 3:36

   What’s Missing Before We Get To AGI?

   1. GTGarry Tan1:53

      So you've been thinking about AGI longer than almost anyone. Uh, when you look at the current paradigm, large scale pre-training, RLHF, chain of thought, how much of the final architecture for AGI do you think we already have, and what's fundamentally missing right now?

   2. DHDemis Hassabis2:08

      Well, first of all, thank- thanks, Garry, for that great introduction, and it's great to be here. Thanks for, for welcoming here. It's amazing space, actually. I'll have to come back here often. Very inspiring that you all get to work in, in, in this space. So the question is, uh, I think the, the components that you just mentioned, I'm pretty sure will be part of the final architecture for AGI. So I think they've come such a long way now, uh, and we've proven out so many things about what they can do. Um, I can't see a world in which we will sort of realize in a couple of years this was a dead end. That doesn't make sense to me. But there still might be one or two things missing on top of, uh, of, of, of what you've, you know, what we already know works. So, um, continual learning, long-term reasoning, uh, some aspects of memory, these are still unsolved. Um, and how to get the systems to be more consistent across the board. Um, I think all of these are gonna be required for AGI. Now, it might be that the existing techniques can just scale up to that with some innovation and some incremental innovation. Um, but it could be that there's still one or two big ideas left, uh, that need to be cracked. I don't think it's more than one or two, if there are out there, and I think, you know, my betting is, uh, about 50/50 if that's the case. So of course, at DeepMind, at, at Google DeepMind, we work on both those things.

   3. GTGarry Tan3:29

      I guess that's so... I mean, working with a bunch of agentic systems, the wildest thing to me is to what degree it's the same weights

4. 3:36 – 6:14

   Why Memory Is Still Unsolved

   1. GTGarry Tan3:36

      ev- over and over. So this idea of continual learning-

   2. DHDemis Hassabis3:38

      Mm-hmm

   3. GTGarry Tan3:38

      ... is so interesting because, like, you know, right now we're sort of cobbling it together with duct tape, you know?

   4. DHDemis Hassabis3:44

      Yes.

   5. GTGarry Tan3:44

      These dream cycles at night and things like that.

   6. DHDemis Hassabis3:47

      Yeah. It's pretty cool, the dream cycles, and we, we used to think about this with consolidation, with episodic memory. Actually, that's what I studied for my PhD is how the hippocampus works and integrates, you know, new knowledge gracefully into the existing knowledge base. So the brain does that amazingly well. It, it, it does it th- you know, during sleep, uh, especially things like REM sleep, replaying back episodes that, that are important so that you can learn from it. In fact, our very first Atari program, DQN, one of the ways it was able to master Atari games was by doing experience replay. So we sort of borrowed that from, from neuroscience and replayed successful trajectories, uh, many times. You know, that's way back in 2013 now, in the, in the dark ages of AI. It was, uh, a really important thing. And, and I agree with you, we're kind of using duct tape right now, so, like, shove it all in the context window. Um, this bit, this seems a bit unsatisfying, right? And actually, even though, uh, we're working on machines, not biological brains, and so potentially you could have, you know, millions or tens of millions size context window or memory, and it can be perfect, there's still a cost to looking it up and finding the right thing, uh, that, that's actually relevant for the specific, uh, decision you've got to make right now, and that's non-trivial, that cost, even if you can potentially store it all. I think there's actually a lot of room for innovation in, in areas like memory.

   7. GTGarry Tan5:12

      Yeah. I mean, the wild thing is it feels like a million- Token context ones is actually bigger than... I mean, it's plenty big, honestly.

   8. DHDemis Hassabis5:18

      Mm.

   9. GTGarry Tan5:19

      You can do so-

   10. DHDemis Hassabis5:19

       Well, it's, it's, it's plenty big for, for, for most things that it should be used for. I mean, if you think about the context window's sort of equivalent to working memory, you know, humans have, we have like a few digits, you know? It's like a, a dozen digits maybe, you know, average of seven. We got million or, you know, 10 million context windows. But the problem is, is that we're trying to store everything in that, you know, things that aren't and not important, things that are wrong. It's pretty brute force currently, and that doesn't seem, uh, right. And then the problem is, if you're now trying to try and process live video, and you're just gonna naively record all the tokens, then actually a million tokens isn't that much. It's only like 20 minutes. So actually you need more if you want something that's gonna understand your, you know, your, what's going on in your life over maybe a month or two.

   11. GTGarry Tan6:07

       DeepMind has, uh, historically leaned into reinforcement learning and search, uh, AlphaGo, AlphaZero,

5. 6:14 – 8:06

   How AlphaGo Shaped Gemini

   1. GTGarry Tan6:14

      and MuZero. Uh, how much of that philosophy is actually embedded in how you're building Gemini toge- today? Uh, is RL still underrated?

   2. DHDemis Hassabis6:23

      Yeah, I think potentially it is. It sort of goes in, in ebbs and wa- waves. You know, we've worked on agents since the beginning of DeepMind. In fact, we als- that's what we said we were working on. So all of the Atari work and AlphaGo most specifically, they're agent systems, and what we meant by that is systems that are able to, you know, accomplish goals on their own, uh, and make active decisions and, and make plans. And so of course, we were doing it in the domain of games to, to, to make it tractable, uh, and then doing increasingly complex games, things like StarCraft after AlphaGo, AlphaStar. So, um, we basically did all the games that were out there. Um, and then of course, the question is, can you generalize those models to be world models or models of language, not just models of simple games, uh, or even complex games? And that's what the last few years has been about. But really, you can think of a lot of the things we're doing today, all the leading models with thinking modes and chain-of-thought reasoning as aspects of what was sort of pioneered with AlphaGo coming back now. And I actually think there's a lot of work we did back then that is relevant today, and we're sort of re-looking at some of those old ideas, um, at scale today in a more general way, including things like Monte Carlo tree search and other, other ways of doing s- augmenting the RL, uh, on, on top of the, the reinforcement learning we're ready to do today. And I think a lot of those ideas, both from AlphaGo and AlphaZero, are really, really relevant, uh, to where we are with today's foundation models, and I think, uh, a lot of that is what we're gonna see of the advances the next few years.

   3. GTGarry Tan7:59

      One question I would have s- like obviously today you need bigger and bigger models to be smarter and smarter.

6. 8:06 – 10:46

   Why Smaller Models Are Getting So Powerful

   1. DHDemis Hassabis8:06

      Mm-hmm.

   2. GTGarry Tan8:06

      But then we're also seeing distillation working.

   3. DHDemis Hassabis8:09

      Yeah.

   4. GTGarry Tan8:09

      And then smaller models can be f- like quite a bit faster, I think. You know, you guys have incredible flash models that are-

   5. DHDemis Hassabis8:14

      Yeah

   6. GTGarry Tan8:15

      ... like nine, like you're finding that they're 95% as good-

   7. DHDemis Hassabis8:18

      Mm-hmm

   8. GTGarry Tan8:18

      ... as, uh, the frontier and at like one tenth the price. Is that right?

   9. DHDemis Hassabis8:22

      I think that's one of our core strengths is, I mean, you have to build the biggest models to, to, to, to have, uh, the frontier capabilities. But I think one of our biggest strengths has been, uh, distilling and packing that power into smaller and smaller models very quickly. Obviously, we, we, you know, we invented the kind of distillation process and, and people like Jeff and Oriol and, and others, and we're still world experts in that. And we also have a huge need to, uh, do it because we've got to serve the biggest probably AI surfaces, um, there are. Obviously, there's search with AI overviews and AI mode, then there's Gemini app, and now increasingly every single product at Google has, you know, Maps and YouTube and so on, has some aspect of Gemini or Gemini-related technology in it. And so that's billions of users, a dozen, more than a dozen billion user products. Um, and they have to be served extremely fast, extremely efficiently and cheaply, and with low latency. So that, that gives us a really important incentive to, to make these flash and even smaller models, flashlight models, uh, extremely efficient. And hopefully that ends up then being really useful for many of the workloads that all of you d- use f- for.

   10. GTGarry Tan9:34

       I'm curious about how much smarter these smaller models can actually be. Like, are there limits to the distillation process? Like could a 50B or 400B model be as smart as like a Mythos for today?

   11. DHDemis Hassabis9:48

       Yeah, I don't, I don't see any, I don't think we've got to any kind of, uh, or at least none of us know yet if we've got to any kind of informational limit. I mean, maybe at some point that will be the case where there's just an information density that can't, we can't get beyond. But I think for now, there's, the, the assumption we make is that, you know, a year later after one of our, uh, leading, you know, pro models or frontier models goes out, half a year later, a year later, you'll have them in the, the really tiny, almost edge models. And you also see some of that goodness in our Gemma models, which hopefully you're all enjoying, our Gemma 4 models, which I think are really amazing power for their sizes. So again, that uses a lot of this, uh, these distillation techniques and, and the idea of how to make things really efficient in these very small models. So I don't really see any limit yet in terms of like some kind of theoretical limit. I think we're still pretty far off of that.

   12. GTGarry Tan10:39

       That's ama- I mean, that is really good-

   13. DHDemis Hassabis10:41

       Yeah

   14. GTGarry Tan10:41

       ... 'cause, uh, you know, one of the weirder things that we're seeing right now is like engineers can do

7. 10:46 – 12:40

   The 1000x Engineer

   1. GTGarry Tan10:46

      like 500 to 1000 times the amount of work that they were doing Like, six months ago, I guess. I mean, the people in this room, they're people who are doing about, like, 1000x the work that, like, I... Steve Yegge talks about this. It's like 1000x the work that a Google engineer from the 2000s was doing

   2. DHDemis Hassabis11:04

      I think it's very exciting. I mean, I think the small models have many uses. One is obviously cost, but the speed can allow, you know, if you think about coding even or other things, you can iterate a lot faster. Also, especially if there's... if you're collaborating with the system, I think there's a, there's a, a, a lot of need for having fast systems, um, that maybe are not quite frontier, like you said, like 95%, 90%, but that's plenty good enough and actually you gain back more than the 10% on the, the iteration speed. So, and then the other big thing I think is running these things on the edge, again, for efficiency reasons, but also for privacy and security reasons too. Um, if you think about different devices that you might run these systems on that per- that, you know, process very personal information. You can also think about robotics as well, um, you know, robots in your house. I think you're gonna want very efficient, uh, very powerful, uh, local models, which maybe are orchestrated, you know, with some bigger models, frontier models that are in the rec- in the cloud, but you only delegate to that in certain circumstances and perhaps you, you know, you process all of the audio visual feed, let's say, locally, and that stays local. I could imagine, uh, that would be a very good sort of, um, end state.

   3. SPSpeaker12:19

      YC Startup School is back. We're hand-selecting the most promising builders in the world and flying them out to San Francisco for July 25th and 26th to discuss the cutting edge of tech. Apply now for a spot. Okay, back to the video.

   4. GTGarry Tan12:33

      Going back to context and memory, models currently stateless, but, you know, continual... Like, what would the developer

8. 12:40 – 13:32

   Continual Learning and the Future of Agents

   1. GTGarry Tan12:40

      experience even be like for someone who's using a continual learning model? Like, you know, any idea, like, how you'd steer it?

   2. DHDemis Hassabis12:46

      I think it's really interesting. I think that's one of the... not having continual learning currently is one of the things holding back agents from doing full, uh, tasks. You know, I think they're really useful for aspects of tasks right now, and you can patch them together and do some really cool things, but they don't adapt well with the context that you're in, and I think that's the missing piece from them being really kind of fire and forget and they'll figure it out themselves. You know, I think they need to be able to learn, um, about the specific context, um, that you're gonna put them in. So, um, I think we have to crack that to get full general intelligence.

   3. GTGarry Tan13:26

      Where are we on reasoning? So models can do really impressive chain of thought now, but they still fail on things a

9. 13:32 – 15:33

   Why AI Still Fails at Basic Reasoning

   1. GTGarry Tan13:32

      smart undergrad wouldn't. What specifically needs to change, and what progress do you expect in reasoning?

   2. DHDemis Hassabis13:37

      There's a lot of, uh, innovation left in, in think- the thinking paradigms, I would say. Again, I think we're fairly, we're doing fairly simplistic things, fairly brute force. Um, one could imagine, uh, I think there's a lot of scope, for example, in monitoring the chain of thought, maybe interjecting midway through, uh, a thought process. I often get the impression with our systems and, and our competitor systems that they're almost overthinking. They're almost getting into sort of loops of things. Like, one thing, uh, I sometimes like to do is p- is play chess against Gemini, and, you know, it's... All the leading foundation models are pretty poor at games, which is quite interesting. It's very, uh, uh, uh, cool to kinda look at the thinking traces, 'cause obviously these are... can be a well-understood... You know, I can tell quite quickly if it's going off on a tangent, and it's very sort of provable what the, what the, the thinking is doing, uh, whether it's useful or not. And so what we see is that, you know, sometimes it will, it will, it will consider a move, it will realize it's a blunder, but it can't find anything better, so it kind of goes back to that move and does it anyway. So it... you know, you just shouldn't be seeing that, uh, uh, happening in a, in a very precise reasoning system. So there's just sort of huge gaps, I think, still, but it may only be one or two tweaks that are required to fix those kind of gaps, just to be clear. But I think that's pretty, pretty obvious they are there, and that's why you get this kind of jagged intelligence. You know, on the one hand, it can solve gold medal problems in IMO, which is super hard, but on the other hand, as we've all seen, it can still make basic elementary maths errors if you pose the question in a certain way, right? So... Or elementary reasoning errors. So there's just something to me about the almost an introspection about its own thought process that I feel like there's, there's something maybe missing there.

   3. GTGarry Tan15:26

      Agents are really big. Some would say they're hyped. I personally think they're just getting started.

   4. DHDemis Hassabis15:31

      [laughs] Yep.

   5. GTGarry Tan15:31

      It's totally insane. What

10. 15:33 – 18:31

    Are Agents Overhyped or Just Getting Started?

    1. GTGarry Tan15:33

       does DeepMind's internal research tell you about where agent capabilities actually are right now versus, you know, sort of the hype out there?

    2. DHDemis Hassabis15:40

       I think we are... I agree with you. I think we're just at the beginning. You have to have an active system, uh, that can actively solve problems for you to get to AGI. That was always clear to us. So agents are that path, and I think we're just getting going. I think all of us are getting used to, how do we best work? And you're leading the way in a lot of this in your own personal experiments, and I'm sure many of you are doing that. I think, how do you incorporate it into your, uh, workflow in a way that isn't just, um, sort of a nice-to-have, but actually starting to do fundamental things? My impression is, at the moment, we're all expe- you know, we're experimenting on lots of things, but we're only in the, maybe the last couple of months starting to find the really valuable places, and the technology's probably only getting good enough for that to be the case, right? Where that it's not a kind of toy, um, nice demonstration, but actually really adding value to your, to your, um, to your time and efficiency. Um, I'd often wonder... I see a lot of people working on, uh, like, setting off y- you know, dozens of agents for, like, 40 hours, but I'm not sure I've seen the output that yet of that quite justify that level of input going in. But I, I think it will come, so I still think we're in the experimentation phase. We haven't seen a triple-A game that tops the app store charts that was sort of vibe coded yet, right? I've seen, and I've programmed, and I'm sure many... We've all done little nice demonstrations, and it's, like, amazing. I can do a prototype of a theme park in half an hour now, which took me six months back when I was 17. It's kind of mind-blowing, and I, and I wish I... I got this feeling if I spent the whole summer working on it, you could make something really incredible. But it still needs craft and, you know, human sort of soul into it and taste. I think that's, that's something that can, that's you have to make sure you still bring that to, to whatever it is you're building. And I think it still shows like it's not quite there yet, because why haven't we seen, uh, a kid making a hit game that's, that sells 10 million copies, right? That should be possible given the effort that's gone in. So something's still somehow missing. Maybe it's to do with the process or maybe it's to do with the tools. I'm not quite sure. You will probably know better than me 'cause I'm sure you're all experimenting on that. But I haven't seen the result yet, which I would expect once this is really delivering that full value, which I think will come in the next six to 12 months.

    3. GTGarry Tan17:57

       Some of it is like how much of it will be autonomous versus... I mean, I don't think we'd see autonomous first. We would actually probably see people in this room-

    4. DHDemis Hassabis18:05

       Yeah

    5. GTGarry Tan18:05

       ... operating at 1000x and then-

    6. DHDemis Hassabis18:07

       That's what you should see first, and then many of you, you know, there'll be like games companies or, you know, other types of companies that have built some kind of best-selling app, best-selling game using, uh, these tools. That's what you should see first, and then more of that will get automated.

    7. GTGarry Tan18:25

       I mean, some of it is like there's a human in there, and then the human doesn't wanna say that the r- the, the agents

11. 18:31 – 20:26

    Can AI Become Truly Creative?

    1. GTGarry Tan18:31

       did it yet.

    2. DHDemis Hassabis18:31

       I think part of it might be though that, um, this... If we wanna discuss like creativity, what I often say about that is like if we look at the things we've done like AlphaGo. So obviously very famously, you'll all know about the move 37 in game two, and for me, I was waiting for a moment like that to start the science projects like AlphaFold. So we started AlphaFold like the day we got back from Seoul, which is 10 years ago now. In fact, I'm going to Korea after this to celebrate the 10-year anniversary of AlphaGo. But it's not enough to come up with move 37, like that's pretty cool, very useful. Um, but can it invent Go? That's what I w- I want a system that can invent Go if you give it a high-level description. You know, like a game you can learn the rules of in five minutes, but it takes many lifetimes to master. It's beautiful aesthetically, um, but you can play it in a few hours in an afternoon. So, you know, maybe you could imagine that would be the high-level description I would give, and then I'd want the, the return, the thing I get back is Go, right? And, um, clearly today's systems I think can't do that. So the question is why? Um, and I think there's something still missing there.

    3. GTGarry Tan19:42

       Well, someone in this room might, might make it.

    4. DHDemis Hassabis19:44

       Then the answer would be there's nothing missing, it just was the way we were using the systems, and that might actually be the answer. It might be that our today's systems c- are capable of that with a brilliant enough creative person using it and providing that impetus, that s- the soul of the project, and being able to probably being, uh, au fait enough with the tools to like almost be at one with the tools. I could imagine that would be happening if you experimented with the tools all day and all night, like probably many of you are doing, that, and you combine that with proper deep creativity, um, something, you know, more incredible could be done.

    5. GTGarry Tan20:19

       Switching gears to open source, I mean, or open, open and ac- open weights. I mean, the recent release of Gemma-

    6. DHDemis Hassabis20:25

       Yeah

    7. GTGarry Tan20:25

       ... you're

12. 20:26 – 22:26

    Open Models, Gemma, and Local AI

    1. GTGarry Tan20:26

       making highly capable open and accessible ones that can actually run locally. What do you think that means for... Will AI be something that is in the hands of the users instead of primarily in the cloud? And does that change who gets to, you know, build with these models?

    2. DHDemis Hassabis20:42

       We're huge proponents of, in general, of open source and open science, and you mentioned AlphaFold at the beginning. You know, we put that all out there for free, and all of our science work, even still today, we publish in, you know, the big journals. We wanted to create, uh, world-leading models for their, their sizes, right? And so that's what hopefully we've done with Gemma and we're, you know, very committed to that path, and hopefully you'll experiment and build and, and enjoy using Gemma. I think it's been like 40 million downloads now, and, uh, just in, you know, two and a half weeks. So we're really excited about that, and I also think it's important for there to be Western stacks on open source. You know, obviously a lot of the Chinese models are excellent and, and they're currently w- were leading in open source, and we think Gemma's very competitive for its sizes, uh, uh, in, in all those respects. And for us, I mean, there is a question of resources, talent, and compute. Like nobody has enough spare compute to just make two, you know, uh, frontier models at maximum size, right, with different attributes, so that's pretty difficult. But also f- for what, for now what we've, we've decided is that our edge models, the things we wanna use for Android and glasses and robotics, um, it's best that they're open models because they're vulnerable anyway on the sur- once you put them out on the surfaces. So they might as well be actually fully open, right? So we've sort of made a decision to kind of unify that, uh, at the, at the kind of we call it nano size level. And so that actually works for us, uh, strategically as well. Um, and you know, we hope as many people as possible build on it, and of course we'll be building on that too.

    3. GTGarry Tan22:19

       Earlier, uh, before we came on, I got to show you a demo of, uh, my version of Samantha from Her, which is-

    4. DHDemis Hassabis22:25

       Yes

    5. GTGarry Tan22:25

       ... uh, harrowing

13. 22:26 – 24:08

    Why Gemini Was Built Multimodal

    1. GTGarry Tan22:26

       for me to try to demo something to you.

    2. DHDemis Hassabis22:28

       Yeah. Very cool.

    3. GTGarry Tan22:28

       Um, and it worked, which is amazing.

    4. DHDemis Hassabis22:31

       Yes.

    5. GTGarry Tan22:31

       Gemini was built multimodal, and I spent a lot of time with a bunch of the models and-

    6. DHDemis Hassabis22:35

       Mm-hmm

    7. GTGarry Tan22:35

       ... I mean, the depth of the context and the tool use with speech directly to model-

    8. DHDemis Hassabis22:41

       Yeah

    9. GTGarry Tan22:41

       ... like there's nothing like Bar none, like the best one, actually

    10. DHDemis Hassabis22:44

        Yeah. Yeah, I think, I think that's a sort of still a slightly underappreciated aspect of, of, of the Gemini series is we, we started it being multimodal from the start. That made it a little bit more difficult actually to begin with 'cause than just focusing on text, for example. But, uh, we believe we're gonna gain from that in the long run, and I think we're seeing that now for, uh, uh, things like world model building, so stuff like Genie that we build on top of Gemini. I think it's gonna be really important for things like robotics, so this is why Gemini Robotics, which many of you probably played around with, I think it's gonna be built on multimodal foundation models, the robotics models, and, uh, we think we have a sort of competitive advantage with, with Gemini being so strong at multimodal. We're using it increasingly in things like Waymo, um, but also if you imagine devices and assistants, uh, that digital assistants that come with you into the real world, you know, maybe on your phone or glasses or some other device, um, it needs to understand the physical world around you and intuitive physics, uh, and the, and the, and the physical context you're in, and that's what our systems are extremely good at, and I think you found that's why you've enjoyed using it in your setup. We're planning to continue on that, and I think we're far and away the strongest models on, on those types of, uh, problems.

    11. GTGarry Tan24:01

        So the cost of inference is, uh, dropping fast. What becomes possible when inference is essentially free, and how does that

14. 24:08 – 25:24

    What Happens When Inference Gets Cheap?

    1. GTGarry Tan24:08

       change what your team is actually optimizing for?

    2. DHDemis Hassabis24:11

       Yeah, I'm not sure inference will ever be essentially free. I mean, there's sort of Jevons paradox and other things about, like, I think we'll just end up using... all of us will end up using whatever we can get our hands on, and you could imagine, uh, m- millions of agents, swarms of agents working together on things. That's one way to use the inference. Or you could imagine, uh, single agents or group- smaller groups of agents thinking for in multiple directions and then ensembling that. So we're experimenting with all these things. Probably many of you are. All of that will use up any inference I think that's available. I mean, one day maybe it can be almost cost zero. Certainly the energy, if we solve fusion or, you know, superconductors or, you know, optimal batteries or some set of those things, which I think we will do with material science, energy costs will be essentially zero. But there'll still be the physical creation of the chips and other things. There'll some, there'll be some bottleneck, um, at least for the next few decades, I think. And so if that's the case, there'll still be rationing on the inference side. You'll still have to use it, I think, efficiently.

    3. GTGarry Tan25:17

       Yeah. Well, luckily, the smaller models are getting smarter and smarter, which is fantastic.

    4. DHDemis Hassabis25:21

       Yep.

    5. GTGarry Tan25:21

       Uh, we got a lot of bio and

15. 25:24 – 28:24

    From AlphaFold to the Virtual Cells

    1. GTGarry Tan25:24

       biotech founders in the audience. I can see a few. AlphaFold 3 took us beyond proteins to a broad spectrum of biomolecules. Uh, how close are we to modeling full cellular systems, or is that still a fundamentally harder problem in a class of its own?

    2. DHDemis Hassabis25:38

       Well, at Isomorphic Labs, which we spun out from, from, from, from DeepMind after we did AlphaFold 2, um, it's, it's which is going amazingly well. It's, it's, it's trying to build out, uh, not just AlphaFold. It's just one piece of the drug discovery process, uh, as many of you know. But we're trying to do the c- the adjacent biochemistry and chemistry to design the right compounds with the right properties and so on. We'll have some big announcements for, you know, very soon to talk about on, on that front. I think that's going really well. Eventually, you want a whole virtual cell. So I've talked about this in many of my science talks about a full working simulation of a cell that you can perturb, and then the, you know, the, the outputs of that would be close enough to experimental that it's useful, right? You could skip out a lot of the, the search steps and generate lots of synthetic data to train other models that then would predict things about, you know, real cells. And, um, I think we're about 10 years away probably from something like a virtual cell, like a full virtual cell. You know, we're starting out, this is we're working on the DeepMind side, science side on a, you know, virtual nucleus, cell nucleus first, 'cause relatively self-contained. The trick with all of these things is can you pick, uh, a slice of the complexity? You know, eventually wanna, wanna model a human body, but can you model it down to the right level of detail? And what slice can you, uh, take out of it that will be self-contained enough you can kind of model and approximate the inputs and outputs into that self-contained system and then just focus on the self-contained system? So a, a nucleus is quite interesting from that perspective. Um, then the other issue is just there's not enough data yet, so you need data. Uh, and I talked to various, you know, top scientists about who work on electron microscopes and other imaging things. If we could image a live cell without killing the cell, that would be, um, game-changing obviously, 'cause then you could convert it into a vision problem, which we would know how to solve, right? But at the moment, there are at least, I don't, I'm not aware of any techniques that can give you a kind of, you know, nanometer resolution, uh, but without destroying, but i- in, you know, in a live dynamic cell, so you can see all the interactions, right? You can take sp- static images at that resolution, obviously, um, really detailed now, and that's quite exciting. But it's not enough, uh, to turn it just into, uh, just into a complex vision problem. So that's one way it could be solved. So it could be a hardware-driven, data-driven solution, or it could be that we build better, uh, s- learned simulators of, uh, these dynamical systems. So that's, that's the more modeling way of solving it.

    3. GTGarry Tan28:17

       Uh, you've been looking at all kinds of science, not just bio. Uh, there's material science, drug discovery, climate modeling, mathematics.

16. 28:24 – 30:43

    AI as the Ultimate Tool for Science

    1. GTGarry Tan28:25

       If you had to rank which scientific domain will transform the most dramatically in the next five years, what's in your list?

    2. DHDemis Hassabis28:30

       Well, they're all so exciting, and that's why, I mean, that, that for me has been my main passion and always the reason why I've worked on AI for my whole career for 30-plus years now is to use AI as the ultimate tool. I always thought AI would be the ultimate tool for science and to advance s- advance scientific understanding, scientific discovery, and things like medicine and just our understanding of the universe around us. So actually when you mentioned our original way we used to articulate our mission statement, which is still, uh, the way we think about it, is there was two steps to it. One was, step one was solve intelligence, i.e., build AGI, and then step two was use it to solve everything else. We had to change that a bit over time because people were like, "Do you really mean solve everything else?" And we did mean that, and I think people are sort of understanding what that means today. But specifically, I was meaning solve other, what I call, root node problems in science. So areas of science that would unlock whole new branches or avenues of discovery, and AlphaFold is the prototypical example of what we want to do. So over three million researchers around the world, pretty much every biology researcher in the world, uh, uses AlphaFold now, and I was told by some of my, you know, pharma executive friends that, you know, almost every drug discovered from now on will have used AlphaFold at some point in it, in the dr- drug discovery process. So that's something we're very proud of, and it's the sort of impact that we hope to have with, with AI. But I do think it's just the beginning. Uh, I, I don't really see any area of science or engineering that this won't be able to help, be helpful with. And the ones you mentioned, I think we're at almost like an AlphaFold one moment, so it's, we've got very promising results, but it's not quite solved the, the grand challenge yet in that domain. But I think we're gonna have a lot to talk about in the next couple of years on all those areas you mentioned, materials, which I'm, I think is very exciting, all the way to mathematics.

    3. GTGarry Tan30:18

       In, in science, I mean, it feels Promethean. It's like, here is this capability and, you know-

    4. DHDemis Hassabis30:23

       I think so. I mean, of course, along with that, incl- including what the, the, the parable of Prometheus, we have to also be careful with how we use that and what we use it for, and also the misuse, uh, that can happen with those same tools.

    5. GTGarry Tan30:36

       A lot of people in this room are trying to build companies applying AI to science. For them, what's the difference between a startup that actually advances

17. 30:43 – 33:30

    Advice for Founders

    1. GTGarry Tan30:43

       the frontier in your view versus one that's just wrapping an API around a foundation model and calling it AI for science?

    2. DHDemis Hassabis30:49

       Well, look, I think there's... One of the things I would recommend, I'm trying to think about, and I think you mentioned this to me before, what would I do today myself if I was sitting in your place in Y Combinator, you know, looking at things? One thing you have to do is obviously intercept where the AI tech is going, so that's one hard part of it. But I do think there's huge, uh, scope for combining where AI is going with some other deep technology area. I just think that that sweet spot is, is whether it's materials or medicine or other really hard areas of science, um, I think that those kinds of interdisciplinary teams, especially if it involves the world of atoms as well, um, there's not gonna be a shortcut to that, uh, at least in the foreseeable future. Those are areas that are pretty safe from just getting swamped by whatever the next update is to the foundation models. So I think if you're looking for things like that, that's one of the more defensible areas I would say, and I've always loved deep tech, so I'm kind of biased towards deep tech things. I think, um, nothing that's really long-lasting or worthwhile is easy, and so I'm always being drawn to, to deep technologies. Obviously, AI was like that back in 2010 when we started out, right? It was, it was thought to just, we know, we know it doesn't work kind of thing is what I was told by investors and even in academia, it was considered to be a very niche subject that we sort of tried in the '90s and we know doesn't work. But if you, you know, if you have belief and conviction in your idea why it's different this time or what special combination from your background that you had, ideally you're expert in both those areas, both the machine learning and the other area you're applying it to, or you can create a founding team with that expertise. I think there's huge impact to be made there and huge value to be built there.

    3. GTGarry Tan32:34

       That's a really important message. I mean, even-- I mean, it's hard, it's easy to forget, like basically once you've done it, you've done it, but before you've done it, people are arrayed against you.

    4. DHDemis Hassabis32:43

       Oh, sure. I mean, no one believes in it, which is why I think you've gotta, you've also got to work in things that you're genuinely passionate about. Like for me, I would have worked on AI- No matter what happened. I just decided from a very young age it was the thing that, um, could be the most consequential thing I could think of. It's turned out that way, but it might not... Maybe we would've been 50 years too early, and it was also the most interesting thing I could think of working on. And so I would still be working on AI today, even if we were still, you know, in a little garage somewhere and it still wasn't quite working. I would've still been trying to find, maybe I'd have been back in academia or something, but I would've found some way of, of continuing to work on it.

    5. GTGarry Tan33:23

       So, I mean, AlphaFold was like an example of a spike that you pursued and it worked. You know, what makes a scientific domain

18. 33:30 – 35:20

    The AlphaFold Breakthrough Pattern

    1. GTGarry Tan33:30

       ripe for an AlphaFold-style breakthrough? And is there a pattern, a certain-

    2. DHDemis Hassabis33:34

       Yeah

    3. GTGarry Tan33:34

       ... objective function? Like-

    4. DHDemis Hassabis33:35

       The way I, I'm, I should write this up at some point when I have five minutes spare, but the lesson I've learned from all the Alpha projects we've done, specifically AlphaGo and AlphaFold, is, um, I think the techniques we have and the problems I look like to look for are great in if the, if the situation can be described as massive combinatorial search space, the more massive the better in some ways, so no brute force or special case algorithm will, will solve it, and that's true of Go moves and of, you know, different configurations of proteins, far more than the atoms in the universe, both of those. And then, um, you have a clear objective function, so, you know, you could think of it as minimizing the free energy in the proteins or, you know, the winning the game of Go. So you need to be able to, you need to specify your objective function clearly so you can hill climb. And then, um, enough data and/or simulator that can generate you, uh, lots of, uh, in distribution, uh, uh, sy- synthetic data. If those things are true, then I think, um, with today's methods, you can go a long way into tackling and finding the kind of needle in the haystack that you need, uh, to, for the solution that you're trying to look for. And I think of just drug discovery, by the way, in the same way, right? There is a compound out there that would solve this disease if one could find it, if one could only find it, right? And that wouldn't have any side effects and so on. And, uh, as long as the laws of physics allows it, then the q- only question is how do you find it in an efficient way, in a tractable way? I think we showed for the first time actually with AlphaGo that these systems could, uh, find those kinds of needles in a haystack. In that case, you know, the perfect Go move.

    5. GTGarry Tan35:14

       I guess, uh, to get a little meta, I mean, we've, we're talking about humans using these methods to create AlphaFold, but then

19. 35:20 – 37:59

    Can AI Make Real Scientific Discoveries?

    1. GTGarry Tan35:20

       there's a meta level, which is humans using AI to explore the space of possible hypotheses. How close are we to AI systems that can do genuine scientific reasoning, not just pattern matching on data?

    2. DHDemis Hassabis35:32

       I think we're close. Um, we're working on these general systems like the, like this thing, we, we have this p- system called co-scientist, and we have other algorithms like AlphaEvolve that can go a little bit beyond what the basic Gemini will do, and obviously all the frontier labs are experimenting in this way. I've yet to seen anything so far, and we, we all tinker with the same things, you know, some math problems that are a little bit harder than IMO and so on. I haven't seen anything yet, um, that is a true, genuine, you know, massive discovery. That's my personal opinion. I think it's coming. I think it may be related to, uh, this earlier qu- this thing we discussed about creativity and, and actually going on beyond the bounds of what's known. So clearly that's just not pattern matching at that point 'cause there's, there is no pattern to match to, and it's a bit more than extrapolation. It's some kind of analogical reasoning, and I don't think these systems have that, or at least we're not using them in the, in the right way to do that. So the way I often say that in science is, can it come up with a hypothesis that's really interesting and not just solve one? When I say just, we're now talking about just like solving the Riemann hypothesis or something. This would be obviously amazing, or one of the Millennium Prize problems, and maybe we're a couple of years out from doing that. Um, but I'd like to solve P equals NP. That's, that's my favorite one. But can you... But even harder than that would be to come up with a new set of, of Millennium Prize problems that were regarded by top mathematicians to be as, you know, deep and meaningful and worthy of lifetime of study and effort to solve, right? I think that's another level harder. And, uh, we don't have, um, you know, I still don't think we know how to do that. I don't think it's, it's magical though. I do think these systems will be eventually be able to do that. Maybe we're missing one or two things. And then the way we would test that is, you know, I sometimes call it my Einstein test, which is, you know, can you train a system with the knowledge of cutoff of 1901, and then will it come up with, you know, what Einstein did in 1905, including special relativity, you know, his Annus Mirabilis. Can, can it, can it do that, right? Uh, and then I think we could run that test. May- maybe we should just run that test and keep seeing if that's possible. And once that is, then I think we're on the verge of these systems being able to invent something new, truly novel.

    3. GTGarry Tan37:52

       So last, last question. For the people who are deeply technical in this room who want to work on something,

20. 37:59 – 40:56

    What to Build Before AGI Arrives

    1. GTGarry Tan37:59

       you know, even close to the scale that what you've created with the, you know, it's one of the largest AI efforts in the world, and you've been a pioneer for all these years. So for that, I think everyone in this room thanks you and the folks at DeepMind very, very deeply from the bottom of our hearts, thank you. What's the thing that you know now about building at the frontier that you wish you'd known at 25?

    2. DHDemis Hassabis38:20

       I think we covered some of it in terms of actually you, you work out that going after hard problems and deep problems, um, is no more difficult in some ways than, than going after a shallower, simpler, more superficial problem. They're, they're, they're just differently difficult. There's different things that are hard about each of those things. But I think given Life's very short, and y- you know, you only have so much time and energy. You might as well put your life force into something that will really make a, a, a difference if you hadn't done it, if you hadn't been there to push it. So I would just think of it through that lens. And then the other thing is, if you're, if you are... And then we talked about deep tech, and I love interdisciplinary, uh, work, and I think that's gonna be even more prevalent in the next few years in combinations of fields and, uh, uh, finding the, the, the, the connections between those fields, and it's gonna be even easier to do that with AI. And then the only other thing I would say is if, you know, if you have your, depending on what your AGI timeline is, you know, mine's like 2030 or something like this, then if you start off on a deep tech journey n- today, usually that you're talking about a 10-year journey for, for true deep tech, in my opinion. So then now you have to just consider AGI appearing in the middle of that journey. So what does that mean? It doesn't... It's not bad necessarily, but you have to take that into account, right? To will it be able to leverage it? What will the AGI system do with it? And it goes a little bit back to what you said earlier about AlphaFold and general AI systems. So one thing I can think, see happening is Gemini, Claude or one of these general systems making use of AlphaFold, like specialized systems as tools. I don't think we're gonna have it just in one giant brain 'cause it will have too much regression in. If I put all the proteins into, you know, o- o- our Gemini, that wouldn't make sense. We don't need Gemini to do protein folding. Going back to your information efficiency, it would definitely affect its language skills or something like that, right? In a bad way. So much better, I think, is to have really good general purpose tool usage models that will then... M- maybe they could even train those specific tools, but they would be in a separate, uh, uh, system. So I think that's kind of interesting to think through the implications of that and then what you might build today. Also, physical things too, like what kinds of factories would you build, what sorts of, um, you know, finance systems, and so on. So I just think you need to really take that seriously in, in, in, on the one hand is like, and imagine what that world would look like, and then build something that would be useful if that comes in halfway through.

    3. GTGarry Tan40:53

       Demis Hassabis, everyone. [clapping]

Episode duration: 40:56