Episode 15 - Inside the Model Spec

Wolfe emphasizes the spec is primarily for humans—users, developers, policymakers—to understand intended behavior; it doesn’t attempt to document every system component (e.g., product features like memory, policy enforcement layers).

OpenAI expects gaps between written intent and model outputs because training is complex and outputs are non-deterministic; they close gaps via training interventions, evals, and sometimes revising the spec if the “violation” reflects a better policy.

When instructions conflict, the model should prefer OpenAI-level policies over developer messages over user prompts, but OpenAI tries to keep many policies low-authority so users can override defaults (tone/style) without breaking safety boundaries.

Because many decisions are ambiguous (e.g., honesty vs kindness), the spec uses borderline scenarios and idealized responses to clarify decision boundaries and convey nuanced “how to talk” guidance that’s hard to formalize.

A key surprise was confidentiality interacting with developer goals in a way that could encourage covert pursuit of developer intent; OpenAI revised the spec so honesty more clearly outranks confidentiality to avoid incentives for deceptive behavior.

With deliberative alignment, models are trained not just to mimic compliant outputs but to understand policies and resolve conflicts; this tends to generalize better, benefiting even smaller models when they have adequate reasoning capability.

Wolfe notes that behavior alone can look like an innocent mistake, while internal reasoning may reveal strategic misbehavior; OpenAI aims not to over-supervise chain-of-thought so it remains a candid diagnostic signal.