No Priors Ep. 59 | With Sarah Guo & Elad Gil

(instrumental music) . Today on No Priors, we are going to have a host-only discussion. There's so much going on over the last couple of weeks in AI. We just thought it would be good to take a big deep breath and a step back, and talk through some of the really big changes that seem to be happening in the landscape. Sarah, there's been a lot of new models that have come out over the last, even just week or two. There's Claude, Grok, Databricks, a variety of folks have launched things. What do you think? What's going on?

Yeah, I think it's a huge update, um, for most people's priors versus a year ago, right? I think it's very, um, it's very likely at this point that you end this year with a handful of GPT4-level models, and that some of those are open source, right? And so, I think Mistral first, but then also, um, Databricks with DBRX, they- they changed the point of view on what you can do with, you know, a relatively small amount of compute, tens of millions of compute, and then also, uh, from a, from a scale perspective. The Databricks team in particular just declared a, uh, a very strong point of view that they call Mosaic's Law, where a model of a certain capability will require a quarter of the, um, you know, dollar capital investment every year due to a bunch of, uh, improvements on the, um, hardware and algorithmic side. And I don't know if that's grounded, uh, in any particular technical belief, but I- I- I- I do think that the model landscape completely shifts versus what people expected to be... I think most people expected it to be quite, um, monopolistic or at least oligopolistic a year ago, right? And I- I think that the... There's still a really big question at the state-of-the-art, because if you go up one level of scale in terms of, um, capital investment if you're still, you know, the dominant factor is- is compute, compute scaling, um, I think that question remains. But there's an awful lot you seem to be able to do with the GPT4-level model. So, I think, like, the net impact of that, uh, is pretty good from the application or the sort of enterprise adoption side.

Yeah, it definitely feels like, um, you know, the most cutting edge, smartest models in some sense are gonna end up with an oligopoly at least in the next couple of years, just because of the scale of capital needed. But also, just how far ahead you start to be as you have a model that can help you build the future models, right? Even just things like data labeling or certain forms of reinforcement learning through AI feedback or other things like that. And so, as you get better and better model capabilities, you start bootstrapping the next generation of models, although obviously you have to do other breakthroughs to- to get there. Um, and then to your point, I think under that, you have this broader swath of different models and companies and things that are available. And one could argue part of what that's gonna do is just kind of flip some of the- the value capture, the revenue, the margin, the people, whatever metric you want to use over to the clouds, because they're gonna be hosting all these things, right? So, whether it's LLaMA or whether it's Claude or whether it's one of these other entrants, there's just gonna be a lot of room, I think, for the clouds to make money over time as well, which I think is a little bit under-discussed in terms of, you know, who captures value in this market besides the model providers. Related to the clouds, um, how do you think about the recent Inflection-Microsoft deal?