No Priors Ep. 35 | With Sarah Guo and Elad Gil

(music plays) Hi, No Priors listeners. Time for a host-only episode. This week, Elade and I talk about the path to better model quality from here, the potential of fine-tuning, RLHF, RLAIF, RAG and retrieval systems generally, Meta's sponsorship of the open source model ecosystem, and finally, the beginning of a new set of consumer applications and social networks. Thanks for tuning in. So one thing everybody is thinking about, uh, is what it takes to get to 10X or 100X better AI systems. Like, I-I think it'd be useful just to sort of enumerate the- the elements to sort of step function better. Elade, what do you think?

Yeah. You know, it's interesting because there's- there's a few different aspects of that that people always talk about. There's scalability of datasets and compute and parameters and all these things. But the reality is, I think a lot of people believe that in order to 10X or even 100X use cases and usages for AI, outside of that there's things that could just be done on existing models today. So you don't need to wait for GPT-7 or whatever. You could start with GPT-4 or GPT-3.5 and add these things. And I think they are kinda bucketed into five or six areas. Number one is multimodality. So that means being able to use text or voice or images or videos, both input and output. So you should be able to talk to a model, type to it, upload an image and ask about the image (laughs) , and then it could output anything from code to a short video for you. Um, second is long-context windows. So basically when you prompt a model, you basically are feeding it data or commands or other things, and everybody realizes that you need longer and longer and longer context windows. So Magic, for example, is doing that for code. You know, you should be able to dump in an entire code repo into a coding model instead of having to do it piecemeal. Um, third, which we're gonna talk about today, is model customization. So that's things like fine-tuning, something known as RAG, r-, uh, there's data cleaning, there's labeling, there's a bunch of stuff that just makes models work better for you. Uh, fourth is some form of memory, so the AI actually remembers what it's doing. Uh, fifth is some form of recursion, so looping back and reusing models. And then sixth, which is related, is potentially a bunch of small models that are very specialized being orchestrated by a central model or sort of AI router that says, "Well, for this, for this specific task or use case, I'm gonna route the prompt or the data or the output into this other model that's doing this other thing," which is basically how the human brain works, right? You, uh, process visual information through your visual cortex, but then you use other parts of your brain to make decisions, right? And so it's very similar to what evolutions were decided was an optimal approach. But I think it's really interesting because I think many people in the field know that these five or six things are absolutely coming, and they- they can dramatically improve the performance on existing systems. Again, 10X, 100X better for certain things. And so it's more just a matter of when, right? It's not really an if anymore. A bunch of people are working on different aspects of this, and, you know, I think it's all coming really fast. And so what... you know, there's sort of two things that came out in the last week or two that are really relevant to this, it'd be great to get your thoughts on. One is OpenAI announcing that they're not gonna allow people to fine-tune models, and the second is Google, where they looked at human-generated feedback versus AI-generated feedback for models and sort of fine-tuning models that way. So I don't know if you wanna tell people a bit more about what happened with OpenAI and why that's important.