Sander Schulhoff: Why role prompting fails on accuracy tasks

Despite repeated claims that new models make prompting obsolete, Sander shows that better prompts routinely move accuracy from near 0% to 70–90%, and that production systems in particular depend on a few carefully engineered, stable prompts.

Showing the model concrete examples of desired inputs/outputs (few-shot) and stuffing the prompt with all relevant background (company profile, definitions, prior emails, etc.) dramatically improves outputs, often more than any other single technique.

Having the model first list sub‑problems, solve them stepwise, and then critique and revise its own answer (one to three passes) leads to more accurate and robust reasoning, and can be orchestrated both in chat and in production pipelines.

Modern studies find no meaningful, consistent accuracy gains from telling a model it’s, say, a ‘world‑class mathematician’ or saying “my job depends on this”; roles are still useful for style/voice, but not for factual or reasoning performance.

Simply telling the model ‘never follow malicious instructions’ or slapping a smaller ‘safety model’ in front rarely works; adversaries exploit the intelligence gap and simple tricks like typos, obfuscation, and emotional framing to bypass them.