How I AISpec-driven development: the AI engineering workflow at Notion | Ryan Nystrom
CHAPTERS
- 0:00 – 3:11
From “I don’t know what I’m doing” to one-shotting features with a spec
Ryan opens with his newest workflow: start from an empty markdown doc, talk through how a feature should work, and have Codex turn that into a spec and then working code. The framing sets the theme of the episode—AI doesn’t just speed up coding, it changes the entire engineering loop.
- •Use candid prompts ("explain like I’m five") to force clarity and reduce confusion
- •Start with a spec before touching code to improve agent success rate
- •Voice-to-text (Whisper) as a fast way to draft intent and requirements
- •Treat the spec as the artifact that enables near one-shot implementation
- 3:11 – 4:52
AI has disrupted a 12-year engineering routine (and made work fun again)
Ryan describes how the last year has radically changed his toolchain and daily habits, after more than a decade of stable workflows. Despite the rapid change, he finds it energizing—more joy, more experimentation, and faster output.
- •Frequent switching of IDEs/terminals/tools as AI workflows evolve
- •Fear and uncertainty coexist with excitement and freshness
- •Increased productivity can feel energizing rather than draining
- •AI’s impact is as much about workflow redesign as tool adoption
- 4:52 – 6:23
Project Afterburner: cutting CI time to one-quarter
Ryan introduces Afterburner, Notion’s effort to dramatically speed up CI. Even as a non-infra specialist, he’s brought in to push improvements by pairing clear goals with a highly AI-driven, fast-moving team culture.
- •CI speed as a key lever for developer experience and iteration velocity
- •Setting an aggressive goal (reduce CI to ~25% of current time)
- •Non-experts can still lead infra improvements by focusing on outcomes
- •Culture: a “fast and AI-pilled” org can apply momentum to DevX work
- 6:23 – 7:24
Daily standups without the boredom: automated pre-reads and agenda-first meetings
Instead of low-signal status round-robins, Ryan runs daily standups where the meeting doc starts blank and is filled automatically with the prior day’s real activity. The team spends the meeting time on decisions, blockers, risks, and next steps—not repetitive updates.
- •Auto-generated meeting pages populated with real work context
- •Shift from “what I did” to problem-solving and decision-making
- •Higher quality, higher frequency meetings without extra overhead
- •Meeting notes become a shared, structured record of outcomes
- 7:24 – 12:54
The custom Notion AI agent that surfaces everyone’s work (Slack, PRs, tasks, metrics)
Ryan walks through how the agent compiles context: Slack activity, closed tasks, merged PRs, telemetry, and yesterday’s transcript. The result is a detailed pre-read that makes wins visible, prompts deeper discussion, and reduces the chance that important work goes unnoticed.
- •Aggregate multiple sources: Slack, Notion tasks, PRs, transcripts, metrics
- •Standard output sections: CI speed, decisions, progress, bugs, questions, risks
- •Democratize visibility so quieter engineers’ impact is equally surfaced
- •Use the pre-read as a conversation starter to drill into notable changes
- 12:54 – 15:01
Burnout protection via less prep and fewer context switches
Claire and Ryan connect the automation to sustainability: leaders can code up until the meeting, avoid constant report-writing, and reduce cognitive load. The biggest win isn’t just time saved—it’s eliminating the mental drain of routine synthesis work.
- •Meeting prep time eliminated; leaders regain maker time
- •Agents provide just-in-time updates without human coordination overhead
- •Reduced context switching protects focus and lowers stress
- •Small daily savings (e.g., ~20 minutes) compound and reduce toil
- 15:01 – 16:32
Why engineering managers (and execs) should write code now
Ryan argues that AI makes it easier than ever for line managers to stay hands-on—fix bugs, make optimizations, and stay close to the work. Claire extends the point to directors/VPs/CTOs: this era rewards hard skills and tool literacy.
- •Managers can contribute safely (not necessarily on P0 hero projects)
- •Staying close to code improves judgment, velocity, and team support
- •AI lowers the barrier to meaningful technical contribution
- •Leadership effectiveness increasingly depends on technical fluency
- 16:32 – 25:08
Building the ‘Hot Potato’ agent: instructions, subagents, and permissions
Ryan shows the internal setup of the scheduled agent: detailed instructions, use of subagents (MapReduce-style fan-out), and a strict permission model. He also demonstrates practical “agent ops,” like letting the agent help configure its own Honeycomb query via screenshot.
- •Scheduled trigger (daily) plus clear time window (last 24 hours)
- •Subagents for parallel retrieval (metrics, Slack, tasks, meeting context)
- •Template-driven output formatting and tone guidance (brief + fun)
- •Permission scoping: read-only broadly, edit only for meeting pages
- •MCP/telemetry integration (Honeycomb) to pull live CI metrics
- 25:08 – 26:38
Boxy: VM-based background agents invoked from Notion comments (@mention Codex)
Ryan introduces Notion’s internal system (“Boxy” / “Software Factory”)—a fleet of VMs running tools like Codex and Claude Code. Engineers can trigger work from a Notion task comment, letting agents implement changes without tying up local environments.
- •Background agent + VM strategy as a scaling mechanism for AI coding
- •Invoke Codex from where work is tracked (Notion tasks/comments)
- •Reduces local setup and frees engineers to parallelize work
- •Pairs naturally with fast CI and preview environments
- 26:38 – 29:22
A real example: shipping a UI fix from a friend’s text to PR + preview URL
Ryan demonstrates an end-to-end flow: write a few sentences and attach a screenshot, then @mention Codex to implement. The agent returns a PR, testing notes, preview link, screenshots of UI verification, and iterates on feedback like type fixes and merge conflicts.
- •Minimal input (a few sentences + screenshot) can be enough to start
- •Agent produces PR + preview URL and documents how it tested changes
- •UI verification via screenshots generated by the agent
- •Review becomes conversational: ask “why” and request fixes directly
- •Agents handle iterative cleanup (types, conflicts) quickly
- 29:22 – 32:06
Old world vs new world code review: psychological safety and model ‘personality’
Claire and Ryan discuss how code review feels different with agents: it’s easier to admit confusion and ask for explanations. They compare Codex’s bluntness with Claude’s warmth and highlight how “explain it to me” prompts improve understanding and outcomes.
- •Lower social friction: easier to say “I don’t get it” to an agent
- •Prompting for ELI5 explanations accelerates learning in unfamiliar domains
- •Different agent “tones” change the developer experience
- •Code review shifts toward inquiry, rationale, and verification
- 32:06 – 36:56
Spec-first development in the repo: agent specs as the new source of truth
Ryan explains Notion’s shift away from bloated prompts toward a spec library checked into the codebase. Engineers draft a markdown spec (often from a voice “yap session”), iterate on it, then point Codex at the spec to implement and verify the feature—often in a single pass.
- •Agent harness redesign: move from giant system prompts to progressive disclosure
- •Store specs in version control under a dedicated folder
- •Draft specs quickly via voice notes → Codex converts to structured format
- •Specs include code pointers and explicit verification steps
- •“Build it from the spec” enables near one-shot implementation
- 36:56 – 39:54
Verification as the real engineering work (and why CI speed matters even more with agents)
Ryan argues engineers are evolving into systems thinkers focused on correctness loops: tests, tooling, and the ability for agents to validate behavior directly. He and Claire connect this to CI: faster pipelines dramatically increase throughput for both humans and swarms of agents.
- •Engineering focus shifts to architecture + verification loops
- •Build tooling (e.g., CLI) so agents can run and test the system end-to-end
- •Specs double as changelog: track how behavior evolves over time
- •Slow CI becomes a bottleneck when agents generate many PRs
- •Fast CI tightens iteration loops and enables high-volume agent output
- 39:54 – 47:53
Lightning round: why Codex, and how to prompt when models go off the rails
In closing, they cover Ryan’s preference for Codex (durable long-running tasks, simplicity, one-shot orientation) and practical prompting tactics. When the model is wrong or sycophantic, Ryan pushes back and demands a defended argument with evidence.
- •Codex advantage: maintains coherence over long, grinding tasks
- •Workflow preference: run multiple agents in parallel vs tight interactive iteration
- •Codex as strong code/security reviewer and good fit for project-based work
- •Anti-sycophancy technique: force the model to defend its claims
- •Direct steering (interrupt, reject, restate constraints) when outputs derail
