By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

Sequoia Capital

Factory's Matan Grinberg: The Coming ‘Dark Factory’ Where Software Builds Itself

Factory started building fully autonomous coding agents in April 2023, two years before enterprises were ready. Matan Grinberg now says this is indistinguishable from being wrong.

Matan GrinbergguestPat GradyhostSonya Huanghost

Jul 21, 202651mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

CHAPTERS

Jump to publisher timestamps

Chapter summaries

Create obsessed customers: output metrics over “customer obsession”
Matan frames Factory’s operating principle as creating customers who become obsessed, not merely measuring internal “customer obsession.” He uses analogies to coaching and performance metrics to emphasize outputs over intentions.
- •Matan argues “customer obsession” is an input metric and can be misleading
- •The goal is to build something so good customers become obsessed
- •Basketball coaching analogy: don’t optimize for “sweating,” optimize for scoring points
- •Factory’s focus is on outcomes that reflect real customer delight
Why enterprises fear vendor lock-in in AI coding tools
Pat asks what makes Factory special in a crowded autonomous coding market. Matan explains that enterprise buyers are deeply wary of any single point of failure—especially after cloud-era lock-in experiences—and want modularity and the ability to swap models.
- •Factory started building fully autonomous agents before enterprises were ready
- •Enterprises want model independence to avoid a single provider controlling their fate
- •Cloud-contract renewal “scars” shape enterprise behavior toward AI vendors
- •Developers and enterprises prefer modular systems they can customize and hot-swap models
- •Factory positions itself as providing frontier models’ benefits without direct model-lab lock-in
“No lock-in” to Factory: keeping automations and artifacts in the customer’s codebase
Pat challenges whether Factory simply becomes the new lock-in point. Matan responds that Factory’s work products—automations, skills registry outputs, and artifacts—remain in the customer’s codebase rather than being held as proprietary “tribal knowledge.”
- •Pat raises the concern: enterprises could get locked into Factory instead
- •Matan says automations and artifacts live in the customer’s codebase
- •Factory avoids withholding organizational knowledge on its side
- •He frames this as part of building trust and sharing optimizations (not extracting them)
Being early is the same as being wrong: the hard two years building ahead of the market
Pat asks about Factory’s “two years in the desert,” after Matan’s comment that being early is equivalent to being wrong. Matan describes the difficulty of selling a future developers weren’t ready for, learning enterprise sales, and confronting slow adoption and model limitations.
- •Matan describes starting the company after dropping out of a PhD and recruiting a high-caliber team
- •Early enterprise and developer readiness for autonomous agents was low
- •Models were less performant, but Matan emphasizes behavioral readiness mattered a lot
- •Developers “vote with their feet,” making adoption signals blunt and unforgiving
- •He underscores there’s no reward for being early if it doesn’t land in the market
Refunding revenue: admitting the product wasn’t good and preserving credibility
Matan recounts realizing Factory had achieved revenue through sales strength even though developers weren’t happy with the product. To avoid churn and maintain trust, the company proactively refunded customers—an emotionally and operationally difficult reset.
- •Factory reached revenue while Matan believed the product still wasn’t good
- •He calls selling without developer love a “ticking time bomb” leading to churn
- •Factory chose to refund customers to align with “create obsessed customers”
- •Matan describes the investor and team stress of voluntarily taking revenue to zero
- •The rationale: preserve credibility so customers would later believe “this is the real deal”
Keeping the team together through rock-bottom moments
Pat asks how Factory held together after giving money back. Matan attributes resilience to ruthless early hiring for mission obsession, tight feedback loops, and candid acknowledgment of how bad things were—building enduring bonds and organizational grit.
- •Matan says the team stayed because hiring prioritized deep mission alignment
- •He emphasized shared ownership: success wasn’t on one person alone
- •Honesty about hardship helped maintain trust internally
- •He likens the period to intense training/rowing: pain builds bonds
- •Matan contrasts Factory’s early scrappiness with competitors that started as instant “unicorns”
The Droid CLI breakthrough: meeting developers where they are
Sonya asks what changed, probing Matan’s claim that better models weren’t the only driver. Matan says the prior interaction pattern demanded too much behavioral change, and the turning point was launching Droid CLI—an interface aligned with existing developer workflows alongside improving openness to agentic work.
- •Fully autonomous agents required too big a workflow leap for most developers early on
- •Matan marks Droid CLI’s launch as a key moment because it fit existing workflows
- •He says enterprise developer openness to new tooling changed significantly over time
- •Guardrails and intuition for using agents became more common inside organizations
- •He notes influence dynamics: trusted voices can accelerate behavior shifts
Defining “frontier performance”: benchmarks, Terminal Bench, and short benchmark half-lives
Sonya asks what “frontier performance” meant for Droid CLI. Matan discusses benchmark churn, contrasts earlier SWE-bench limitations with Terminal Bench’s broader terminal-based tasks, and notes how quickly benchmarks get absorbed and saturated.
- •Matan says good benchmarks get “benchmaxed” within months
- •He contrasts SWE-bench (more limited) with Terminal Bench (more end-to-end terminal tasks)
- •Terminal Bench measured tasks beyond file edits, like broader development workflows
- •He notes later models began scoring extremely high, shrinking benchmark usefulness
- •He implies frontier claims must evolve as benchmarks quickly become obsolete
Harness tactics that keep agents strong: caching, compaction, and tool-use validation
Sonya asks what makes a great harness beyond benchmark gaming. Matan details tactics like token caching, handling long sessions through compression/compaction, preserving “needle in the haystack” information, and using tools/environment checks to validate work.
- •Token caching rate materially affects cost/performance dynamics
- •Compaction/summarization quality determines long-session reliability
- •Needle-in-haystack retention is a critical compaction stress test
- •Tool use and environmental validation help ensure work is correct
- •Factory uses internal benchmarks to compare out-of-the-box agents vs its harness
Why multi-model harnesses beat model+harness co-design
Matan challenges the intuition that owning the model plus harness yields best results. He argues a harness exposed to many models avoids overfitting to one model’s quirks and can generalize better, sometimes outperforming a model vendor’s own harness on that vendor’s model.
- •Matan says co-design isn’t necessarily superior for harness quality
- •He uses an analogy: training on broader data beats training only on personal data
- •He argues more model exposure makes the harness more robust and less overfit
- •He claims Droid sometimes outperformed vendor tools on benchmarks with new model releases
- •He notes vendor incentives may favor “better together,” but reality favors multi-model robustness
Concrete model-behavior differences and a harness example: to-do lists and drift control
Pat asks for an illustration of why multi-model harnessing helps. Matan compares different model “personalities” and explains how harness design must normalize user experience—using to-do list tracking as an example of preventing drift and preserving steps through long sessions and compaction.
- •Matan contrasts models as more “friendly/wandering” vs more “OCD/methodical,” affecting harness needs
- •Some harnesses enforce strict adherence to a generated to-do list to prevent wandering
- •Users want consistent experience even when switching models
- •Losing to-do structure can become a failure mode during long sessions and compaction
- •A robust harness preserves task structure so steps don’t disappear mid-migration
From token-maxing to cost rationalization: routing models by task and constraints
Sonya shifts to token economics, describing a move from maximizing usage to rationalizing cost. Matan outlines adoption phases that led to wasteful usage and positions Factory’s router as a way to dynamically select models based on task needs, organizational policies, and reliability requirements—using natural-language routing instructions.
- •Matan describes organizational phases: pressure to adopt, then performance-review-driven token maxing
- •He cites examples of expensive models being used for trivial questions
- •Factory’s router routes tasks to different models depending on need and constraints
- •Routing can vary by team, codebase type, or reliability requirements (generate/test/review with different models)
- •Routing procedures can be expressed in natural language and informed by internal benchmarking
Open models catching up and the forecast for open-model token share
Pat asks about open-model progress and which is best; Matan points to a specific open model as strong and says Factory internally uses open models heavily. He argues open models should be compared to “frontier minus one,” predicts open models will become the majority of token share, and distinguishes token share from the most valuable, high-leverage work.
- •Matan names a specific open model he says is “incredible” and notes heavy internal use
- •He argues comparisons should be to frontier-minus-one because open models lag by a generation
- •He expects open models to dominate token share due to cost and optionality
- •He emphasizes token share differs from “leverage share”: a small fraction of tokens may be highest value
- •He frames intelligence allocation as automating low-leverage work to expand time for high-leverage thinking
Pricing beyond tokens: usage-based today, outcome-based later and a task marketplace concept
Sonya asks about business model and pricing as open models change unit economics. Matan says customers currently want usage-based pricing, seat-based doesn’t fit Factory, and outcome-based pricing may emerge later—possibly via a marketplace where model providers bid to complete scoped tasks with explicit validation criteria.
- •Matan says usage-based pricing aligns incentives today; seat-based doesn’t make sense for Factory
- •He expects a long-term shift toward outcome-based pricing, but not yet for enterprise readiness
- •He describes a marketplace-like future where model providers bid on tasks at fixed prices
- •Validation criteria become central to define whether tasks succeeded
- •He notes open questions: task subdivision/scoping and operationalizing outcome measurement
The software factory vision: codifying tribal knowledge, mapping workflows, and closing feedback loops
Sonya asks about the “software factory” concept and how close the industry is to it. Matan argues every company already has a software factory—just an inefficient one—and that the opportunity is to codify tribal knowledge, map how features ship, and connect work to business outcomes so leaders can allocate dollars between headcount and tokens with rigor.
- •Matan says large organizations have feature-release processes involving many people that aren’t even well-understood
- •He argues tribal knowledge, approvals, and checklists can be codified and automated
- •A software factory clarifies inputs (signals) and the assembly line for building features
- •Closing loops enables measuring whether work actually changed retention/usage or other outcomes
- •He predicts future budget decisions will compare incremental dollars to headcount vs tokens and require quantitative insight
AI transformation playbook: hackathons, permission to fail, and change driven from within
Pat asks for techniques customers use to mature and reinvent for AI. Matan highlights company-wide hackathons, cultures that tolerate failure, bold willingness to reconsider “sacred cows,” and change that originates from technical leadership and individual contributors rather than being imposed by boards.
- •Company-wide hackathons create time and social permission to build with AI
- •Organizations that accept early failures learn faster and adopt more effectively
- •Matan cites a large customer example as being proactive about not being late to the shift
- •Teams succeed when they’re willing to revisit entrenched processes and restore them if needed
- •Transformation works better when driven internally by tech leadership/ICs than externally by boards
Async agents and the “dark factory”: autonomous tokens and optimism about long-term impact
Sonya asks for predictions over the next 12 months; Matan predicts a major shift from synchronous, user-triggered usage to asynchronous agent activity. He ties this to the “dark factory” idea—software work happening continuously without direct prompting—and closes with an optimistic view that while short-term turbulence is likely, better tooling will help engineers solve more important problems with higher-quality software.
- •Matan predicts most tokens will shift to asynchronous usage in 12–24 months
- •He describes agents proactively acting on signals (e.g., customer issues) rather than waiting for prompts
- •He connects the “dark factory” metaphor to autonomous, always-on software production
- •He expects short-term turbulence from resource misallocation and painful corrections
- •He rejects the idea engineers disappear, arguing more problems can be solved with software as productivity rises

Publisher timestamps

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.