Mark Zuckerberg — AI will write most Meta code in 18 months

(instrumental music plays) I would guess that the world is gonna get a lot more... a lot funnier and, like, weirder. If you think that something someone is doing is bad, and they think it's really valuable, most of the time, in my experience, they're right and you're wrong.

I am worried that we're just removing all the friction between getting totally reward hacked by our technology.

We are trying to build a coding agent that advances LLaMA research. I would guess that, like, sometime in the next 12 to 18 months, we'll reach the point where, like, most of the code that's going towards these efforts is written by AI. I tend to think that, for at least the foreseeable future, this is gonna lead towards more demand for people doing work, not less. If you've gotten the cost of providing that service down to one-tenth of what it would have otherwise been, maybe now that actually makes sense to go do.

All right, Mark, thanks for coming on the podcast again.

Yeah, happy to do it. Good to see you.

You too. Last time you were here, you, um, you had launched LLaMA 3.

Yeah.

Now you've launched LLaMA 4.

Well, the first version.

That's right. What's new? What's exciting? What's changed?

Oh, well, I mean, uh, the whole field's so dynamic, so, I mean, I, I feel like a ton has changed since the last time that we talked. Um, Meta AI has almost a billion people using it now, monthly, so that's, um, that's pretty wild. Um, and, you know, I think that this is gonna be a really big year on all of this, because, um, especially once you start getting the personalization loop going, uh, which we're just starting to build in now really, um, from both the context that all the algorithms have about what you're interested in feed and all your profile information, all the social graph information, but also just what you're interacting with the AI about. I think that's just gonna be kind of the next thing that's, um, that's gonna be super exciting, so really big on that. The modeling stuff continues to make really impressive advances too, as, as you know. Um, the LLaMA 4 stuff, uh, I th- I'm pretty happy with the first set of releases. You know, we announced the, um, we announced four models, and we released the first two, the Scout and Maverick ones, which are kind of like the mid-size models, mid-size to small. Um, it's not like... You know, actually, the most popular LLaMA 3 model was, um, was the, the eight billion parameter model.

Mm.

So we're, we, we've, we've got one of those coming in the LLaMA 4 series too. Um, our internal code name for it is Little Llama.

(laughs)

But, um, but that, that's, that's, that's coming probably, you know, over, over the next, over the coming months. But the, um, the Scout and Maverick ones, um, you know, I mean, they're good. They're, they're some of the highest intelligence per cost that you can get of any model that's out there, natively multimodal, very efficient, run on one host, um, designed to just be very efficient and low latency for a lot of the use cases that we're building for internally, and, you know, that's our whole thing. We, we basically build what we're, what we want, and then we open source it so other people can use it too. So I'm excited about that. Um, I'm also excited about the Behemoth model, which is, is coming up. Um, that's gonna be our first model that is, uh, sort of at the frontier. I mean, it's like more than two trillion parameters, so it is... Y- y- I mean, it's, you know, as, as the name says, it's g- it's quite, quite big, um, so we're kind of trying to figure out how we make that useful for people. It's so big that we've had to build a bunch of infrastructure, um, just to be able to push train it ourselves, and we're kind of trying to wrap our head around how does the, like, like, the average developer out there, how are they gonna be able to use something like this, and how do we make it so it can be useful for, um, distilling into models that are of reasonable size to run? 'Cause you're, you're obviously not gonna wanna run, um, you know, something like that in a, in a consumer model. But, um, but yeah. I mean, it's, there, there's a lot to go. I mean, as, as you saw with the, with the LLaMA 3 stuff last year, the initial LLaMA 3 launch was, um, was exciting, and then we just kind of built on that over the year. 3.1 was when we released the 405 billion model. 3.2 is when we got all the multimodal stuff in. Um, so I, we basically have a roadmap like that for this year too, so a lot going on.