No Priors Ep. 54 | With Sarah Guo & Elad Gil

(instrumental music) . Today on No Priors, we're having a special episode of Sarah and me just talking. Hello, Sarah. How are you?

Hey, Elade. What's going on? I see you a lot.

Not much. Good to see you.

Let's talk about models. What's going on in the model world?

Yeah. Um, I guess there's a lot of hand models that are emerging, so I was thinking of maybe trying to do that-

Mm.

... eventually.

It's almost as good of a business as, as investing.

I know, right? Um, yeah, so there's been a lot that's happened in the model world, uh, recently. Obviously, Google launched Gemini, which I think had a few interesting characteristics both in terms of, uh, performance, but also the huge context window, right? It was a million token context window. Uh, companies like Magic, I think, in the past have actually put out, like, a 5 million token context window model and things like that, but it's really exciting to see that. And I think for certain application areas like biology, longer context windows actually seem to be quite important. And so for example, if you're doing a protein folding model and you have a short context window, you're often actually not encapsulating much of the protein, right? The average, uh, protein is, I think, something like 300 amino acids long, at least in the human genome, but there are things that are dramatically larger than that and so you just can't capture it in some of the context windows being used for biological models. And so I do think this is gonna be one of those areas that will end up being more important than people think, at least in the short run. Um, but Gemini 1.5 seems to have some really interesting performance characteristics. There's obviously Sora from OpenAI, which was, um, the video model that, uh, you know, is just beautiful to watch. You know, there's other model companies like Pika and others that I think are doing exciting things as well. And then, um, Mistral or Le Miz launched, uh, Le Chat, which is really the name of the product.

Le Big Model.

Le Big Model. Le Big Mac.

I believe they call it Mistral Large. Yes. (laughs)

Yes. Le Large. Mistral Large. (laughs) They launched that, and the thing that's really, really impressed me about Mistral is just the velocity of shipping. It's incredibly impressive. They went from basically starting the company to almost GPT4 level in less than a year, right?

Nine months, yep.

It's amazing. And they have, uh, you know, small performant models. They have Le Big Mac or, you know, the large model. They have chat. They have multiple languages. It's just- it's very impressive execution. So... And then I think the other thing that they just launched or announced was that deal with Microsoft where, you know, they're- they're now being licensed onto Azure, and so I think the main models in Azure now are OpenAI, LLaMA, Mistral, and then some of the Microsoft models. So again, that's striking as well. So, uh, just very impressive progress by that company so far.