At a glance
WHAT IT’S REALLY ABOUT
Founders share how Managed Agents accelerate outcomes, memory, and sandboxing
- Three founders describe using Claude Managed Agents to build meeting preparation/task automation, cross-account sales reporting (“Watchtower”), and code+telemetry-driven product analytics that can suggest fixes.
- The discussion highlights three Managed Agents capabilities in practice—outcomes (rubric-based grading), memory (learning org concepts and preferences), and sandboxing (safe source-code access and execution).
- Speakers argue outcomes are critical when wrong results are harmful, enabling independent verification and the option to hide uncertain outputs rather than show false positives.
- They describe rapid iteration cycles (days to weeks) made possible by a managed harness, allowing teams to focus on product experience, prompts/tools/orchestration, and customer learning instead of infrastructure edge cases.
- The group debates evals, model migrations, and cost attribution, noting difficulties evaluating stateful systems (memory, third-party tools) and requesting more levers for batch/flex pricing, latency optimizations, and finer-grained cost telemetry.
IDEAS WORTH REMEMBERING
5 ideasIndependently verified outcomes can be better than always showing an answer.
He said the brief is only useful if it’s correct; a wrong identity match (e.g., the “wrong Tom”) can actively confuse users. Their approach is to have an independent assessor verify the result and suppress output when confidence is low, rather than risk a false positive.
Outcomes work by encoding both correctness and UX expectations into a rubric.
He described outcomes as supplying a rubric and letting the agent iterate until it satisfies that rubric, including objective checks (e.g., correct profile) and presentation/UX constraints (scannable, properly ordered, not overly long). This grading happens before the meeting so there’s time to assess.
Org-wide memory turns messy, company-specific business logic into reusable context.
He described learning company-specific concepts (forecasting definitions, ownership rules, messy CRM field meanings) from user interaction, saving them into org-wide memory, and sometimes asking clarifying questions on first use. This reduces upfront configuration while improving results across the organization.
Managed Agents can power cross-account “fan-out and roll-up” workflows for complex queries.
He explained that for complex cross-account questions, the system can write code, do programmatic tool calls, narrow candidate sets, and fan out work to independent agents per account before rolling results up. This supports queries spanning hundreds of deals/accounts.
Sandboxing enables safe code+telemetry workflows without rolling custom security infrastructure.
He emphasized snapshotting customer source code should be sandboxed due to secrets and safety, and that Managed Agents helped cover edge cases without building bespoke infrastructure. He described multiple session types, including per–pull request reviews, overnight batch detection, and on-demand investigation when users ask questions.
WORDS WORTH SAVING
5 quotesAlmost everybody confines the thinking to literally just the period during it, but the place where agents become the most interesting are thinking about how do we help you prepare for the conversation that you're about to have?
— Unattributed
Briefs are awesome when they're correct, like, if they actually tell you who you're talking to correctly.
— Unattributed
You can just choose not to tell people something when you don't know if your information's correct.
— Unattributed
In the early days, you wanna innovate, you wanna get features in front of customers fast. You wanna be able to change really, really quickly.
— Unattributed
So I think the goal is, you know, h- how do we ultimately to a self-healing piece of software, where as people go fast, as people check in more and more changes, we're seeing what actual user's doing beyond just our eval sets.
— Unattributed
High quality AI-generated summary created from speaker-labeled transcript.
