No Priors Ep.61 | OpenAI's Sora Leaders Aditya Ramesh, Tim Brooks and Bill Peebles

(instrumental music plays) Hi, listeners. Welcome to another episode of No Priors. Today, we're excited to be talking to the team behind OpenAI's Sora, which is a new generative video model that can take a text prompt and return a clip that is high definition, visually coherent, and up to a minute long. Sora also raised the question of whether these large video models are world simulators and applied the scalable transformers architecture to the video domain. We're here with the team behind it, Aditya Ramesh, Tim Brooks, and Bill Peebles. Welcome to No Priors, guys.

Thanks so much for having us.

Thanks.

To start off, why don't we just ask each of you to introduce yourselves so our listeners know, uh, who we're talking to. Aditya, mind starting us off?

Sure. I'm Aditya, I lead the Sora team together with Tim and Bill.

Hi, I'm Tim. I also lead the Sora team.

I'm Bill, also lead the Sora team.

Simple enough. Um, maybe we can just start with, you know, the OpenAI mission is AGI, right? Um, greater intelligence. Is text-to-video, like, on path to that mission? How'd you end up working on this?

Yeah, we absolutely believe models like Sora are really on the critical pathway to AGI. We think one sample that illustrates this kind of nicely is a scene with a bunch of people walking through Tokyo during the winter. And in that scene, there's so much complexity. So you have a camera which is flying through the scene. There's lots of people which are interacting with one another. They're talking, they're holding hands. There are people selling items at nearby stalls. And we really think this sample illustrates how Sora is on a pathway towards being able to model extremely complex environments and worlds, uh, all within the weights of a neural network. And looking forward, you know, in order to generate truly realistic video, you have to have learned some model of how people work, how they interact with others, how they think ultimately, and not only people, also animals and really any kind of object you want to model. And so looking forward as we continue to scale up models like Sora, we think we're going to be able to build these, like, world simulators where essentially, you know, anybody can interact with them. I, as a human, can have my own simulator running and I can go and, like, give a human in, in that simulator work to go do, and they can come back with it after they're done. And we think this is a pathway to AGI which is just going to happen as we scale up Sora in the future.

It's been said that we're still far away despite massive demand for a consumer product. Like, what, uh, is, is that on the roadmap? What do you have to work on before you, you have broader access to Sora? Tim, you wanna talk about it?