Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold

... so I wouldn't be surprised if we had AGI-like systems within the next decade. It was pretty surprising to almost everyone, including the people, uh, who first worked on the scaling hypotheses that how far it's gone. In a way, I look at the large models today and I think they're almost unreasonably effective for what they are. It's an empirical question whether that will hit an asymptote or a brick wall. I think no one knows.

When you think about, uh, superhuman intelligence, is it, like, con- still controlled by a private company?

A- as Gemini are becoming more multimodal and we start ingesting audiovisual data as well as text data, I do think our systems are going to start to understand the physics of the real world better. The world's about to become very exciting, I think, in the next few years as we start getting used to the idea what true multimodality means.

Okay. Today, it is a true honor to speak with Demis Hassabis, who is the CEO of DeepMind. Demis, welcome to the podcast.

Thanks for having me.

First question. Given your neuroscience background, how do you think about intelligence? Specifically, do you think it's, like, one higher level general reasoning circuit or do you think it's thousands of independent subskills and heuristics?

Well, it's interesting because intelligence is so, uh, uh, broad and, um, you know, what we use it for is- is so sort of generally applicable. I think that suggests that, you know, there must be some sort of high level, uh, uh, common things in- i- you know, common kind of algorithmic themes, I think, around how the brain processes the world around us. So, um, of course, there- there then there are specialized parts of the brain that- that do specific things, um, but I think there are probably some underlying principles that underpin all of that.

Yeah. How do you make sense of the fact that in these LLMs though, when you give them a lot of data in any specific domain, they tend to get, uh, asymmetrically better in that domain? Uh, w- uh, wouldn't we expect a sort of, like, general improvement across all the- all the different areas as well?

Well, I think you... First of all, I think you do actually sometimes get surprising improvement in other domains when you improve in a specific domain. So for example, uh, when these, uh, large models sort of improve at coding, that can actually improve their general reasoning. So there- there is some evidence of some transfer, although I think we would p- we would like a lot more evidence of that. Um, but also, you know, that's how, uh, the human brain learns too, is if we experience and practice a lo- a lot of things like chess or, you know, writing, creative writing, whatever that is, we also tend to specialize and get better at that specific thing, even though we're using, uh, sort of general learning techniques and general learning systems in order to, uh, sp- you know, to get good at that domain.