Episode 15 - Inside the Model Spec

Hello, I'm Andrew Main, and this is the OpenAI Podcast. Today, we are joined by Jason Wolfe, a researcher on the alignment team, to discuss the model spec, how it shapes model behavior, and why it's important for anyone building or using AI tools to understand

The, the spec often leads where our models actually are today. At this point, you know, models are pretty good at, like, kind of going out and finding new, interesting examples. Models should think through hard problems. Don't start with the answer, like, actually think it through first.

What'd you do this weekend?

Uh, what did I do? Uh, just, like, kid stuff. I don't even remember what.

Like, did they talk to ChatGPT or...?

Uh, yeah, we use, we use voice mode sometimes. She'll, like, ask it random, like, science questions and, and that kind of thing. It's fun.

Right.

You know, one time she, she snuck in there before I could dive in, like, "Is Santa Claus real?"

Oh, wow.

I was like, "Oh, sh-" Uh, no, no, yeah, the... Luckily, the, the model, uh, answered in a, a way that was spec compliant, which is, you know, to recognize that maybe there's actually a, a kid who's asking this question, and you should kind of, uh, you know, uh, just be a little bit vague, uh, about your answer, so.

So we, we've talked before here about model behavior, and the term model spec has come up numerous times. I would love for you to unpack what that means, model spec.

Yeah. So, uh, the spec is our attempt to explain, uh, the high-level decisions we've made about how our models, uh, should behave. Uh, and yeah, th- this covers many different aspects o- o- of model behavior. A few key things to note that it, it is not. Uh, one, it's not a, uh, a statement that our models perfectly follow the spec today. Uh, aligning models to the spec is, uh, is always, uh, an ongoing process, and this is, uh, you know, something we, uh, we learn about as, as we deploy our models, and we measure their alignment with the spec and, uh, and, you know, understand what users like and don't like, uh, about these, and then come back and, uh, iterate on both the, the spec itself and, uh, and, uh, and our models. Uh, the spec is also not an implementation artifact. So, um, I think this is maybe a, a common confusion that the primary purpose, uh, of the spec is really to explain to, to people how it is our models are supposed to behave, uh, where, you know, the, these people are, you know, uh, employees of OpenAI and also, uh, users, developers, policymakers, members of the public. Uh, it is, you know, a secondary goal that our models are, are, are able to understand and apply the spec. But, uh, we never, uh, kind of put something in the spec or change the wording in the spec in a way where the goal is just to, uh, have this better teach our models.