Skip to content
ClaudeClaude

Memory and dreaming for self-learning agents

How memory and dreaming turn Claude Managed Agents into self-learning systems. This session walks through design considerations for memory architectures and how dreaming verifies and enriches memory between sessions.

May 8, 202624mWatch on YouTube ↗

CHAPTERS

  1. 0:00 – 2:01

    Why memory is the next critical agent primitive

    Mahesh frames memory as the missing piece for long-horizon agents: continuous self-learning and better context management over hours- to days-long tasks. He positions memory alongside earlier Anthropic primitives (MCP, harnesses, Skills) that expanded tool use and capabilities but didn’t fully solve learning across time and sessions.

    • Agents now run for many hours/days, raising the need for long-horizon context
    • Earlier primitives (MCP, Claude Code/Agent SDK, Skills) increased capability but not continuous learning
    • Memory is presented as the foundation for self-learning agents that improve from experience
  2. 2:01 – 3:01

    What agents should learn: tasks, environments, and other agents

    He outlines what “memory” enables: learning task success criteria and strategies, understanding persistent environments like codebases, and sharing learnings across agents. The emphasis is on multi-agent swarms building a shared, evolving world model.

    • Capture success criteria, common mistakes, and effective/ineffective strategies
    • Persist knowledge about environments: codebases, files, assets, operational context
    • Enable cross-agent learning and sharing in multi-agent systems
    • Self-managed memory becomes essential as systems grow in complexity
  3. 3:01 – 4:02

    Managed Agents Memory beta and early customer results

    Mahesh announces Memory in Claude Managed Agents (public beta) and explains its goals: strong defaults for intelligence, multi-agent support, and enterprise-grade control. He cites early impact from customers, including a dramatic reduction in first-pass mistakes.

    • Memory is available in Claude Managed Agents (public beta)
    • Designed to work out-of-the-box while remaining flexible for production
    • Supports many concurrent agents in the same environment
    • Example: Rakuten reported ~90% drop in first-pass mistakes plus efficiency gains
  4. 4:02 – 5:03

    Design requirement #1: maximize intelligence by default (from claude.md to autonomous memory)

    He contrasts earlier, more constrained memory approaches (claude.md notes, explicit memory tool calls) with a more agent-native design. The direction is to delegate more memory decisions to Claude rather than forcing rigid schemas.

    • Early memory: claude.md as constrained self-notes; SDK memory tool as a structured call
    • Trend: reduce over-constraint; let the model manage more of the environment
    • Analogy to Skills: agents can manage virtual environments, so they can manage memory too
  5. 5:03 – 6:34

    File-system memory model and why it fits modern Claude models

    Memory is modeled as a file system with a hierarchy and formats that Claude can edit using familiar tools like bash and grep. Mahesh ties this to Opus 4.7’s strength at file-system-based memory: deciding what to store, how to structure it, and keeping it organized.

    • Memory is represented as files in a hierarchy the agent can update
    • Agents use bash/grep workflows to maintain memory (like agentic coding)
    • Opus 4.7 performs well at choosing what to remember and how to organize it
    • Emphasis on practical organization: splitting files, maintaining structure, updating over time
  6. 6:34 – 8:06

    Design requirement #2: scaling memory for multi-agent concurrency and permissions

    As organizations run hundreds or thousands of agents in parallel, memory must support scoped access and safe concurrent updates. Mahesh introduces permission scopes (read-only vs read-write) and optimistic concurrency to prevent overwrites.

    • Agents need mix-and-match access to multiple memory stores per session
    • Permission scopes enable read-only org knowledge vs read-write working memory
    • Concurrency is critical with many agents touching shared state
    • Optimistic concurrency uses content hashes to prevent clobbering changes
  7. 8:06 – 9:39

    Design requirement #3: production-grade enterprise control (auditability + portability)

    For real deployment, memory must be inspectable, attributable, and governable. Mahesh highlights version history/audit logs, rich metadata about who/when/which session made changes, and a standalone API to integrate with external governance and pipelines.

    • Version history/audit logs for every memory update
    • Attribution metadata: agent identity, timestamps, session details
    • Standalone/portable API for bespoke systems beyond Managed Agents
    • Supports governance needs: PII scanning, cleanup pipelines, cloning/exporting memory
  8. 9:39 – 10:40

    A layered model of frontier memory: storage, structure/content, and process

    He synthesizes the system into layers: where memory is stored (and what metadata exists), how it’s structured (files, procedural memory like Skills), and how updates happen (triggers, frequency, sources). This sets up why a new process layer is needed beyond real-time agent writes.

    • Storage layer: persistence plus metadata/attribution
    • Structure/content layer: file-system memory; Skills as procedural memory
    • Process layer: when/how memory updates happen and what drives changes
    • Managed Agents Memory addresses parts of this, but gaps emerge at scale
  9. 10:40 – 11:11

    Why real-time memory isn’t enough at scale: siloed sessions and missed global patterns

    In multi-agent deployments, agents often miss learnings discovered by others and don’t maintain the global store efficiently. Mahesh describes observed issues: repeated mistakes, shared patterns not captured, and inefficient holistic curation when memory updates are tied only to individual tasks.

    • Agents can miss learnings from other sessions/agents
    • Repeated common mistakes appear across many runs
    • Local task focus leads to siloed, incomplete memory updates
    • Need a complementary process to curate memory globally and efficiently
  10. 11:11 – 12:12

    Dreaming (research preview): batch learning from transcripts to update memory

    Mahesh introduces “dreaming,” a new out-of-band process launching in research preview. Dreaming scans recent agent transcripts to detect patterns, mistakes, and winning strategies, then generates organized memory updates that can be applied automatically or reviewed.

    • Dreaming is launched in research preview in Managed Agents API
    • Batch, asynchronous process operating on recent sessions/transcripts
    • Finds common mistakes, patterns, and effective strategies across runs
    • Outputs an updated/organized memory diff; can be applied immediately or manually reviewed
    • Early result: Harvey saw a 6× task completion rate increase in a legal benchmark scenario
  11. 12:12 – 13:13

    How dreaming runs: scheduling, inputs, and applying updates safely

    He explains operational mechanics: dreaming can run on a cron, via console/API, or when agents finish tasks. It comprehensively reviews transcripts and proposes memory changes that teams can gate with checks or manual review before applying.

    • Triggered periodically (cron), via console, or after task completion/spin-down
    • Uses transcripts from recent sessions to propose improvements
    • Produces updated memory state/diff for the target memory store
    • Supports review and validation workflows via API before applying changes
    • Goal: next-day agents improve from prior-day experience automatically
  12. 13:13 – 17:45

    Why dreaming helps harness design and scaling: separate objectives and amortize compute

    Mahesh argues dreaming is powerful because it’s out-of-band: it can aggregate across agents, preserve clear in-task objectives, and avoid adding latency to live sessions. He frames dreaming as a way to spend extra compute to maintain a large, fresh memory/knowledge base—similar to test-time compute and search indexing amortization.

    • Out-of-band aggregation enables cross-agent pattern discovery beyond single-session perspective
    • Separates ‘memory quality’ objective from ‘task completion’ objective for clearer agent goals
    • No added latency to the hot path of task execution
    • Enables enterprise-scale knowledge bases that remain fresh and token-efficient
    • Analogy: extra compute like test-time compute; also like building a high-quality search index upfront
  13. 17:45 – 22:49

    Putting it together + demo: SRE agents, scoped stores, version history, and dreaming diffs

    In a demo, Mahesh shows an SRE workflow with multiple agents sharing memory stores: an org-wide read-only store and an SRE read-write store. He demonstrates how memory accelerates repeated incidents, how version history and concurrency metadata support reliability, and how dreaming produces curated diffs (pattern notes, deduplication, stale removal, verification).

    • SRE system spins up agents from alerts; agents share multiple memory stores
    • Read-only org knowledge store (runbooks/SLOs/owners) vs read-write SRE store
    • Second agent short-circuits investigation by reading prior notes (efficiency + intelligence gains)
    • Version history shows who changed what/when; precondition hash supports optimistic concurrency
    • Dreaming job selects sessions (e.g., last 7 days), spawns sub-agents to analyze transcripts, outputs diffs
    • Dreaming outputs: new pattern insight (e.g., 60-second retry pattern), deduplication, stale removal, verification note
  14. 22:49 – 24:28

    Closing: memory as a prerequisite for days-long agents and continuous improvement

    Mahesh concludes that as agents run longer, memory becomes a load-bearing part of reliable performance. He encourages builders to start using Memory and Dreaming in the Claude Managed Agents API to enable continuous self-improvement.

    • Agents will increasingly run for days or many hours
    • Memory is central to making long-running, reliable agent systems possible
    • Invitation to build with Memory + Dreaming in Managed Agents API

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.