At a glance
WHAT IT’S REALLY ABOUT
How Warp’s Buzz agent learns from team feedback loops daily
- Warp built Buzz to triage and draft responses for thousands of social mentions monthly, saving a small team time while keeping human authenticity for final replies.
- Prompting alone plateaued at an “80% good” stage because rigid rule-based instructions were brittle and produced robotic outputs in nuanced, taste-based scenarios.
- Switching from detailed if/then rules to flexible principles improved generalization, reduced instruction length, and produced more natural, context-aware drafts.
- To make the agent improve reliably, the team added a “learn how to learn” mechanism that converts human critique into better underlying instructions rather than more one-off rules.
- A low-friction daily feedback loop in Slack (emoji reactions + short notes) lets Buzz compare its suggestions to actual actions, generate takeaways, and open reviewable pull requests to update its skill files safely.
IDEAS WORTH REMEMBERING
5 ideasDesign for continuous improvement, not a perfect first prompt.
Petra’s core message is to prioritize a feedback loop that steadily upgrades the agent as new cases appear, instead of trying to encode everything upfront.
Use principles to capture judgment and taste better than checklists.
Principles (e.g., be empathetic, avoid defensiveness) generalize to new social contexts and reduce robotic, rule-following behavior that breaks on novelty.
Teach the agent how to translate feedback into better instructions.
Without guidance, the agent tends to respond to critique by adding narrow rules; a dedicated “learning” skill asks what instruction changes would have produced the ideal output.
Exploit existing team “breadcrumbs” as training signal.
Emoji reactions and short Slack thread notes already used for coordination become labeled outcomes (what was actually done) that the agent can learn from with minimal extra work.
Keep human-in-the-loop for authenticity while automating the heavy lifting.
Buzz drafts, triages, and explains reasoning, but humans still post final replies—yielding major time savings while preserving a genuine brand voice.
WORDS WORTH SAVING
5 quotesNow, that's a significantly less, uh, hands up. So that gap is what we are going to talk about how to close today and, and help all of you guys, uh, build better agents that actually, um, work for you and do things for you, um, on a daily basis.
— Petra
And that sort of eighty percent there but just not quite is where I think a lot of agents die, and it sort of, um, end up more-- almost worse than if you would not have an agent because you end up spending time on tweaking it and prompting it and spending time on trying to get it right because you can kind of feel that it's almost there, but it's not quite good enough to just let it run on its own.
— Petra
So we ended up with, instead of these long list of rules, we ended up with principles that really much better encapsulated what we wanted the agent to do.
— Petra
The agent sort of needed to learn how to learn.
— Petra
If you remember one thing from this whole talk, if you recall just one thing that you can take into, um, into practice, um, when you, when you go back to your jobs and, and you build your own things, is to focus on designing that feedback loop.
— Petra
High quality AI-generated summary created from speaker-labeled transcript.
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome