a16zWhy OpenAI Is Racing to Make Computer Use Superhuman
CHAPTERS
- 0:00 – 1:19
Why computer use is nearing “superhuman” performance
The conversation opens with the claim that computer-using agents are now faster than the average human and approaching expert-level speed. The hosts set up the main themes: what’s improved recently, where it’s useful, and what’s next as these agents become more capable.
- •A speaker says computer use is already faster than the average human and close to superhuman.
- •The hosts frame the episode around what has been unlocked in recent models and products.
- •The discussion will cover consumer and enterprise implications, plus trust and control.
- 1:19 – 3:24
What Astra changes: speed, creativity, reliability, and alignment
Ari describes Astra as improving computer use “around the edges,” including faster task completion and more reliability. James adds that Astra is the most aligned model, with stronger resistance to prompt injection and unintended actions—especially relevant for enterprise scenarios.
- •Ari says Astra is faster on average due to using fewer thinking tokens for the same task.
- •Ari cites improved creative capability, referencing a demo drawing in Microsoft Paint.
- •Ari describes a meaningful reliability jump that increases trust.
- •James says Astra is the most aligned model and more resistant to prompt injection.
- •James says it’s better at avoiding unintended actions, helping with sensitive enterprise work.
- 3:24 – 5:43
Consumer vs. enterprise: frontier experiences vs. obvious legacy pain points
James contrasts consumer work as pushing the frontier of new experiences, while enterprise adoption is driven by clear needs in clunky, long-lived software. Ari highlights consumer value in offloading annoying daily tasks via a cloud computer.
- •James says consumers are where they explore new experiences enabled by faster models.
- •James says enterprise use cases are often obvious due to legacy software lacking modern integrations.
- •James mentions companies building skills to teach models to operate older UIs better.
- •Ari describes consumer use as saving time on annoying tasks to focus on what matters.
- •Ari describes “Dots,” where each dot has a cloud computer with a browser that can do tasks like booking appointments or buying items.
- 5:43 – 8:05
How the OpenAI team uses Dots: proactive monitoring and end-to-end workflows
James explains why Dots stand out: they can proactively flag issues and handle long-running, variable-shaped tasks. He gives concrete examples spanning email/calendar cleanup and Slack-to-Sentry debugging loops using both cloud and local control.
- •James says Dots can proactively catch things that slip through the cracks.
- •He describes automatically detecting email invites that aren’t on calendars and offering to accept them.
- •He contrasts brittle scheduled automations with higher-level, judgment-based goals for Dots.
- •He outlines a workflow: identify important messages, log into Sentry, pull logs, analyze them, and report back.
- •He emphasizes Dots work well for long-running tasks that aren’t identical each time.
- 8:05 – 17:21
Approaching superhuman: balancing speed, cost, autonomy, and oversight
Ari and James discuss the tradeoffs between speed and observability. As agents get fast enough that humans can’t watch every action, enterprises especially need stronger controls, human-in-the-loop options, and auditability to maintain safety and accountability.
- •Ari says OpenAI focuses on speed because users either wait on results or want to observe correctness.
- •Ari says computer use is now faster than the average human and nearing “superhuman” speed versus expert users.
- •Ari describes multiple models across quality and price points, with cheaper models improving over time.
- •James emphasizes the autonomy vs. safety tradeoff: speed can reduce the ability to observe and audit actions.
- •James describes enterprise controls, including human-in-the-loop settings and auditability as autonomy increases.
- 17:21 – 18:16
Enterprise safety controls: Guardian, permissions, and trust-building UX
James details mechanisms enterprises use to feel safe: deterministic sandboxing and configurable human approvals for sensitive actions. Ari adds product design choices that build trust, such as visible cursor behavior and permission prompts, reinforced by Fabrizio’s UX observations.
- •James describes “Guardian,” a model that evaluates each action for safety and intent alignment.
- •He says enterprises can choose how much is auto-approved versus requiring human approval.
- •James describes deterministic sandboxing: allowlists/denylists for apps and websites.
- •James gives examples of approval triggers like downloads/uploads or visiting a new website.
- •Ari says the product’s visible cursor and permission prompts help users build trust over time.
- 18:16 – 19:12
Agents on the open web: ad incentives and “human + agent” browsing via WebMCP
James explains that open-web dynamics differ from internal enterprise tools, especially for ad-supported sites whose incentives don’t align with agent-driven usage. He describes an approach where agents augment rather than replace browsing, enabled by standards like WebMCP.
- •James says internal company software is more straightforward because automation is desired there.
- •He highlights conflicts for ad-supported businesses when agents consume content without producing valuable ad impressions.
- •He says some sites want to embrace agents and are adopting standards for human-and-agent co-use.
- •He points to WebMCP support in the in-app browser as a way to elevate experiences without replacing sites.
- •He describes a flow where humans browse for enjoyment and hand off tedious steps to the agent, then take over for final decisions like completing a transaction.
- 19:12 – 20:07
Which tasks to hand over: “anything you can do,” plus flexible input and sensitivity boundaries
James argues that, since Astra, the agent is broadly capable of doing what a person can do on a computer, with improved handling of ambiguity. Both James and Ari describe building the habit of reaching for computer use first, while noting cases where sensitivity or simplicity changes the decision.
- •James says he tries to start by asking whether the agent can do the task first.
- •He notes sensitivity boundaries, like wanting personal messages to feel like they come from him.
- •James describes “flexible input,” where the agent asks for details or continues while pulling needed context.
- •Ari emphasizes universality: computer use works with any application without needing connectors or MCP servers.
- •James says computer use becomes a universal capability that may appear as part of complex tasks rather than a separate mode.
- 20:07 – 24:11
Best enterprise use cases: “boring is opportunity” in systems of record
James organizes enterprise opportunities into automating tedious workflows and enabling more ambitious creative work. He lists concrete “boring” examples—ERPs, CRMs, procurement, finance reconciliation, and sales workflows—plus market research tasks where the agent can try products and summarize insights.
- •James frames two categories: tedious workflow automation and more ambitious creative enablement.
- •He highlights systems of record (ERPs, CRMs) as major targets for automation.
- •He gives a detailed procurement example spanning spreadsheets, internal procurement software, and vendor outreach.
- •He describes finance workflows like month-end reconciliation across ledgers when tools lack APIs.
- •He mentions sales prospecting and CRM updates, plus market research where the agent signs up for tools, tests them, and reports findings.
- 24:11 – 25:36
Consumer automation and a new computer interface: voice-driven, fluid collaboration
Ari extends the “boring automation” idea to consumer life—forms, inbox cleanup, spreadsheets—and adds a category he’s most excited about: changing how people use computers entirely. He describes voice interaction and a future where multimodal experiences enable more real-time collaboration with agents.
- •Ari lists consumer drudgery automation like DMV forms and unsubscribing from emails.
- •He shares examples of people doing work they wouldn’t otherwise attempt, like finance or meal-planning spreadsheets.
- •He notes the agent succeeds even in less-common tools like Apple Numbers.
- •Ari predicts a shift toward voice-based computer use and evolving software form factors.
- •He expects more fluid, multimodal, real-time collaboration beyond background task execution.
- 25:36 – 32:38
Will agents run legacy software or replace it? Personalized software and the future of UI
The group debates whether agents will primarily operate existing legacy software or drive new AI-native products. Ari expects both to coexist for a long time; James argues UI won’t disappear because humans value direct manipulation and craftsmanship, and both foresee more personal, niche software experiences.
- •James poses the question: keep legacy software with an agent layer, or replace it with new AI-native software?
- •Ari says legacy systems won’t go away overnight and expects both approaches to matter for a long time.
- •Ari suggests computer use may pressure software to become more agent-friendly, citing WebMCP as an example.
- •James says he’d be sad if the internet became faceless agent-to-agent interactions and is bullish that UI remains.
- •James and Ari discuss a future of highly personalized software—built for small groups or even a single person—while agents assist with summaries and automation.
- 32:38 – 37:30
Advice for founders: pick the right modality, design human-in-the-loop collaboration, and lean into domain depth
Seema asks what founders should do when building with computer use for enterprise workflows. Ari emphasizes that the right human-agent collaboration depends on whether the product is interactive for team members or running in the background via API, while James emphasizes domain expertise, go-to-market execution, and integrating disparate software into a cohesive experience.
- •Ari says product modality matters: interactive tools for individuals vs. background automation in the cloud via API.
- •He highlights design questions: when to notify humans, how to summarize outcomes, and avoiding notification overload.
- •James says there’s still a strong startup opportunity because there are many problems and limited lab bandwidth.
- •He argues moats come from deep industry knowledge, regulatory understanding, go-to-market, deployment, and servicing.
- •James gives an example workflow in restaurant operations that combines vision inputs, procurement actions, and ERP adjustments via computer use.
- 37:30 – 44:13
What’s next: superhuman speed, coherent cross-browser experiences, cloud access, and proactivity
Ari and James close by outlining near-term frontiers: faster performance, multimodal collaboration, and making the overall experience easier to understand. James points to today’s fragmented browser states (extension, in-app, cloud) as confusing, while Ari highlights increasing proactivity so the agent adapts to the user and helps them remember to delegate tasks.
- •Ari reiterates priorities: speed, reaching superhuman performance, and blending multimodal interfaces for fluid collaboration.
- •James says making the experience coherent across multiple browsing contexts is a key product challenge.
- •He describes consumer confusion from separate cookies/passwords across different browsers and cloud computers.
- •Both discuss scaling computer use beyond power users, especially by bringing cloud computers to phone-centric users.
- •Ari says proactivity will help the agent come to the user, reducing the need for users to adapt or remember to use it.
