Dario Amodei — “We are near the end of the exponential”

So we talked three years ago. I'm curious, in your view, what has been the biggest update over the last three years? What has been the biggest difference between what it felt like the last three years versus now?

Yeah. I would say, actually, the underlying technology, like the exponential of the technology has, has gone, broadly speaking, I would say about, about as I expected it to go. I mean, there's like plus or minus, you know, a, a couple-- there's plus or minus a year or two here, there's plus or minus a year or two there. I don't know that I would've predicted the specific direction of code, um, but, but actually when I look at the exponential, it, it is roughly what I expected in terms of the march of the models from like, you know, smart high school student to smart college student to like, you know, beginning to do PhD and professional stuff, and in the case of code, reaching beyond that. So, you know, the frontier is a little bit uneven. It's roughly what I expected. I will tell you, though, what the most surprising thing has been. The most surprising thing has been the lack of public recognition of how close we are to the end of the exponential. To me, it is absolutely wild that, you know, you have peop- you know, within the bubble and outside the bubble, you know, but, but you have people talking about these, these, you know, just the same tired, old hot button political issues and like, you know, ar-ar-around us we're like [chuckles] near the end of the exponential.

I, I wanna understand w-what that exponential looks like right now because the first question I asked you when we recorded three years ago was, you know, "What's up with scaling? How, how does it work?" Um, I have a similar question now, but I feel like it's a more complicated question because at least from the public's point of view-

Yes

... three years ago, there were these, you know, well-known public trends where across many orders of magnitude of compute you could see how the loss improves. And now we have RL scaling, and there's no publicly known scaling law for it. It's not even clear what exactly the story is of is this supposed to be teaching the model skills? Is it supposed to be teaching meta learning? Um, what is the scaling hypothesis at this point?

Yeah. So, so I have actually the same hypothesis that I had even all the way back in 2017. So in 2017, I think I talked about it last time, but I wrote a doc called the, The Big Blob of Compute Hypothesis. And, a-and, you know, it, it wasn't about the scaling of language models in particular. When I, when I wrote it, GPT-1 had, had just come out, right? So that was, you know, one among many things, right? There was-- Back in those days, there was robotics. People tried to work on reasoning as a separate thing from language models. There was scaling of the kind of RL that happened, it, it, that, that, you know, kind of happened in AlphaGo and, uh, you know, that, that happened at Dota at OpenAI and, um, you know, people remember StarCraft at DeepMind, you know, the AlphaStar. Um, so, uh, it was written as a more general document, and, and the specific thing I said was the following: That, and, you know, it's, it's very, you know, Rich Sutton put out The Bi-Bitter Lesson a couple years later, um, uh, but, you know, the, the hypothesis is, is basically the same. So, so what it says is all the cleverness, all the techniques, all, all the kind of we need a new method to, to do something like that doesn't matter very much. There are only a few things that matter, and I think I listed seven of them. One is, like, how much raw compute you have. The other is the quantity of data that you have. Then the third is kind of the quality and distribution of data, right? It needs to be a broad, broad distribution of data. The fourth is, I think, how long you train for. Um, the fifth is you need an objective function that can scale to the moon. So the pre-training objective function is one such objective function, right? An-another objective function is, you know, the, the kind of RL objective function that says, like, you have a goal, you're gonna go out and reach the goal. Within that, of course, there's objective rewards like, you know, like you see in math and coding, and there's more subjective rewards like you see in RL from human feedback or kind of higher order, higher order versions of that. And, and then the sixth and seventh were things around kind of like normalization or conditioning, like, you know, just getting the numerical stability so that kind of the big blob of compute flows in this laminar way instead of, instead of running into problems. So that was the hypothesis, and it's a hypothesis I still hold. I, I don't think I've seen very much that is not in line with that hypothesis. And so the pre-trained scaling laws were one example of what, of, of, of, of, of kind of what we see there. And indeed, those have continued going. Like, you know, uh, you know, I think, I think now it's been, it's been widely reported, like, you know, we feel good about pre-training. Like pre-training is continuing to give us gains. What has changed is that now we're also seeing the same thing for RL, right? So we're seeing a pre-training phase, and then we're seeing like an RL phase on top of that. Um, and with RL, it's, it's actually just the same. Like, you know, e-even, even other companies have, have published, um, uh, um, like, um, you know, in some of their, in some of their releases have published things that say, "Look, you know, we train the model on math contests, you know, AIME or, or the kind of other things, and, you know, how well, how well the model does is log linear in how long we've trained it." And we see that as well, and it's not just math contests. It's a wide variety of RL tasks. And so we're seeing the same scaling in RL that we saw for pre-training.