François Chollet: Why Scaling Alone Isn’t Enough for AGI

I think we're probably looking at AGI twenty thirty. Around the time, uh, that we're gonna be releasing like maybe ARC six or ARC seven. You're not gonna stop, uh, AI progress. I think, I think it's too late for that. And so the next question is, okay, like AI progress is here. Uh, it's actually gonna keep accelerating. How do you make use of it? How do you leverage? How do you ride the wave? That's the question to ask.

[on-hold music] Today, we're lucky to be joined by François Chollet, founder of the ARC Prize, a global competition to solve the ARC-AGI benchmark. His latest project is Ndea, a lab exploring a new paradigm in frontier AI research. François is one of the best people in the world to help us understand the current AI moment and where all of this is going. François, thank you so much for joining us today, and congrats on the launch of ARC-AGI V3.

Thanks so much for having me. I'm super excited to be here. Super exciting time to talk about AI.

So François, tell us a little bit about India. So what exactly is it and what are you guys trying to achieve?

Right. So Ndea is this new AGI research lab, and we are trying some very different ideas. And so our goal is basically to build this new branch of machine learning that will be much closer to optimal, unlike, unlike deep learning.

All of us right now are sort of taken by what's going on with code. Uh, I have sort of this viral moment right now where I got to forty thousand stars this morning-

Oh, wow

... on, uh, GStack. So it's like, oh, this is an open source project that now is one of the biggest ones, and I have more than a hundred PRs from contributors to deal with. I guess you're, you know, one of the best people to talk to about this because you're, you're actually literally coming up with something that is a totally different pathway.

That's right. That's right. So, uh, what we're doing at Ndea is, uh, we're doing program synthesis research. And when I talk about program synthesis, often people ask me, "Oh, so are you doing like codegen? Are you, uh, building an alternative to coding agents?" And it's actually not at all what we are doing. We are working at a much, much more, uh, much lower level than that. Uh, what we're actually doing is that we are trying to build a new branch of machine learning, an alternative to deep learning itself, uh, rather than like coding agents. Coding agents are like this very, very high level last layer piece of the stack, and we're actually trying to rebuild the whole stack on top of different foundations. So we're building a new learning substrate that's very different from, you know, parametric learning, deep learning. So if you go back to, uh, the problem of machine learning, you have some input data, some target data, and you're trying to find a function that will map the inputs to the targets that will hopefully generalize to new inputs. And, uh, if you're doing deep learning, what you're doing is that you have this parametric curve that serves as your, as your function, as your model, and you're trying to fit the parameters of the curve via gradient descent. And this is basically what we are doing, uh, except we are replacing the parametric curve with a symbolic model that is meant to be as small as possible. It's like the simplest, uh, possible, uh, model to explain the data, to model what's going on. Uh, and of course, if you're doing that, you cannot apply gradient descent anymore, so we are building something that we call, uh, symbolic descent, which is like the symbolic space equivalent of gradient descent. The idea is to build this new machine learning engine that's giving you, uh, extremely concise symbolic models of the data you're feeding into it, and then we are gonna make it scale. And so everything you're doing with machine learning today, with parametric curves, we should be able to do it, uh, with symbolic models in the future in a, in a way that will be much, much closer to optimality. Much closer to optimality in the sense that you're gonna need much less data to obtain the models. The models are gonna run much more efficiently at, at inference time because they're gonna be so small. And because they are so small, they will also generalize much better and compose much better. You know, the, the minimum description length principle, that the model of the data that is most likely to generalize is the shortest. And I think you cannot find a model like this if you're doing parametric learning. You need to, you need to try symbolic learning.