No Priors Ep. 74 | With Google DeepMind VP of Research Oriol Vinyals

Hi listeners, and welcome to No Priors. Today we're talking to Oriol Vinyals, the VP of research at Google DeepMind, and technical co-lead for Gemini. His storied career in machine learning includes leading the AlphaStar team, which built a professionally competitive and pioneering StarCraft agent all the way to today. And we're really excited to get his historical perspective on where we are in machine learning. Welcome to the show, Oriol.

Yeah, amazing. Thanks, Tara, for in- the invitation. And likewise, thanks a lot for, for hosting me.

Yeah, thanks for joining.

Last year was an eventful year at Google and DeepMind. Um, you know, how is that research effort organized now, and what do you, what do you think of the mission as internally?

Yeah. So sure, I mean, I'm happy to obviously discuss the different phases that research, uh, organizations have gone through in the last many years. But focusing on last year, two major events happened. One was that, uh, the Gemini project was formed, um, as a result of having two sort of parallel efforts on LLMs, uh, mostly led by, uh, Google Brain and, and what we now call Legacy DeepMind. So, uh, earlier in the year, there was, uh, an effort to merge the two, the two projects, and that's when sort of Geoff, Geoff and I came together and brought the two teams together to create the very first Gemini model, which, uh, was eventually released later in the year. Then the second big event was to, uh, take the, all the organizations, uh, that were doing, uh, AI research or AGI research and also form a singular organization. That's what today is called Google DeepMind. Um, and it comes from, uh, Google Brain and Legacy DeepMind coming again together under one roof. Obviously Gemini being a very large and very important project within that organization. Um, and really the goal, uh, of Gemini itself is to create an awesome core model to power, uh, the technology that of course LLMs today are, um, powering all around the world. And we obviously expect this to all increase.

How do you interact with the rest of the company and, like, Google as a business? And I'm like, I feel like I have to ask you, uh, does AI replace traditional search?

So even running that from a research standpoint, um, is super interesting, right? There's, there's, um, two major centers, one in California, one in London, given the organizations that we come from. So that in itself is very interesting. In a way we, we have the project running 24/7, which is helpful when you train these large models. And, and then you have to do a few things, right? One of the things we do, of course, is trying to build state-of-the-art technology, showing from sort of a research, knowing where the field is coming from and where it's going to, trying to really, um, showcase from our, our own sort of intuitions and ambition what might come next, right? So a, a prime example of this was, for example, the long context that we released earlier in the year, right? Millions, millions of, uh, tokens now are be- being able to be processed by, by our models. But then of course we also, um, sort of take into consideration all the different needs, right? From the different products that we work with. Google has a lot of product areas. So we try to focus, of course, initially especially to form the project, we try to focus on critical projects. And you see that very much, um, by how Gemini's first surfaced to, to users or to enterprises, right? So obviously cloud, um, and enterprise is very important. Developers as well. Uh, super cool to put these models in the hands of creative minds that are gonna do things you, you didn't even anticipate these models could do. Um, and then very important formerly known Bard, now Gemini app, which is sort of the chatbot surface of our models. And then maybe the last, uh, very important piece indeed is, is search, which, uh, is trying to integrate, of course, this technology into their product. Um, and of course has a lot of users. So it's extremely exciting to, to think, well, the decisions you make at modeling, uh, eventually and eventually means just maybe a few couple months after or so will make it into, into the users that maybe are signing up for a beta, et cetera. So, super exciting and it, and it's obviously connected. It's the core of, of the company really, especially for the products that require very intelligent, um, AI systems like the ones we're creating today.