By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

OpenAI’s New Agent Stack: Computer Use, Decisions API, UltraFast, Dots—Ari Weinstein & Nikunj Handa

From giving AI agents their own cloud computers to making Computer Use faster than humans at real-world tasks, OpenAI is pushing agents much closer to actually operating software end-to-end. In this episode, recorded immediately after OpenAI DevDay, Ari Weinstein, who leads product and engineering for Computer Use, joins swyx and Vibhu to unpack Dots, GPT-6.1, the Agents API, App Shots, and the rapid improvements that have made Computer Use dramatically faster, cheaper, and more reliable. Ari explains why Computer Use is now “180 degrees different” from where it was months ago, how agents are learning to debug and recover from failures, why combining screenshots with accessibility data, the DOM, Playwright, and generated code changes the speed equation, and why the next frontier is making agents literally superhuman at using software. In the second half, Nikunj from OpenAI’s API team breaks down the new developer stack: async tool calling, mid-turn steering, WebSockets, UltraFast inference, the Decisions API, prompt caching, pre-warming, compaction, and the Agents API. He also shares the unusually fast story behind Decisions API, how OpenAI’s inference teams are using coding agents to optimize their own systems, and the bigger question of whether OpenAI is effectively building an “AI cloud” with higher-level primitives for agents, memory, compute, and state. We discuss: • Why OpenAI thinks Computer Use has changed dramatically in just the last few months • Dots and what changes when every agent gets its own Linux computer • Why Computer Use can now complete some tasks faster than the average human • The path from human-level to “literally superhuman” software operation • Why modern Computer Use agents are much better at debugging and recovering from failure • How screenshots, accessibility trees, the DOM, Playwright, and generated JavaScript work together • App Shots and why they give models much richer context than ordinary screenshots • Why Computer Use can close the loop between writing software and testing it • Trust, permissions, and safety when agents can make payments or operate third-party websites • Async function calling and why models no longer need to stop reasoning while tools run • Mid-turn steering, WebSockets, and the architecture behind more responsive agents • UltraFast inference and how OpenAI is pushing frontier models toward much lower latency • The rapid internal story behind Decisions API • Why Decisions API is more than just structured outputs at low latency • GPT Live, fast tool calling, and real-time computer control • How OpenAI is using Decisions API internally for support classification and other workflows • Longer prompt caching, cache pre-warming, and building cache-aware applications • Server-side compaction vs manual compaction for long-running agent threads • What should live inside an Agents API versus inside a developer’s own harness • OpenAI as an “AI cloud” and the search for higher-level primitives beyond raw model APIs Sponsorship and business inquiries: business@latent.space — Ari Weinstein • Head of Product & Engineering, Computer Use at OpenAI Nikunj Handa • Head of Product, API at OpenAI Timestamps 00:00:00 Hook 00:01:12 OpenAI DevDay: Dots, GPT-6.1, Agents API, and Decisions API 00:04:04 Dots and Personal Cloud Computers 00:06:11 Why Computer Use Is “180 Degrees Different” 00:07:16 From Sky to Self-Debugging Computer Use Agents 00:10:36 How Computer Use Sees and Operates Software 00:13:21 From Faster Than Humans to Superhuman Computer Use 00:17:15 Agents API: Trust, Permissions, and Safety 00:18:43 Computer Use for Coding, Testing, and QA 00:20:26 GPT-6 APIs, Async Tool Calling, and UltraFast Inference 00:24:33 The Rapid Story Behind Decisions API 00:26:44 What Decisions API Is and How It Works 00:31:36 What OpenAI Is Building With the New APIs 00:33:35 Prompt Caching, Pre-Warming, and API Performance 00:36:32 Context Compaction for Long-Running Agents 00:38:25 Memory, Higher-Level APIs, and the AI Cloud

Ari WeinsteinguestswyxhostVibhuhostNikunj Handaguest

Sep 30, 202640mWatch on YouTube ↗

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

WHAT IT’S REALLY ABOUT

OpenAI DevDay deep dive: faster Computer Use agents and Decisions API

OpenAI DevDay announcements discussed include Dots with personal cloud computers, Computer Use inside the Agents API, App Shots, a new GPT-6.1 Sol model positioned as cost- and speed-advantaged for Computer Use, and the new Decisions API.
Computer Use improvements are framed as a step-change over the past year: agents are better at debugging, can use multiple modalities (screenshots, accessibility, DOM/Playwright), and sometimes write and execute JavaScript to perform many actions in one go.
The speakers describe performance bottlenecks shifting from model speed to end-to-end system latency, including waiting on third-party sites to load and reducing delay between load completion and the next model action.
For developers, the conversation stresses trust and safety considerations—permissions, consent for consequential actions, and limiting site/app access—alongside the practical benefit of building on OpenAI’s distributed harness that models are trained against.
On the platform side, new API capabilities include async tool calling, mid-turn steering via WebSockets, UltraFast inference work, Decisions API for parallel fast classification/decisioning, plus caching, pre-warming, and context compaction for long-running agents.

IDEAS WORTH REMEMBERING

5 ideas

Computer Use is shifting from merely capable to speed-dominant, aiming for “superhuman” software operation.

He says the team is now “hill climbing on latency,” and that faster agents would enable more real-time product experiences and make people default to delegating more tasks instead of doing them manually.

Dots’ personal cloud computers broaden what assistants can do beyond browser-only automation.

He explains each Dot has access to its own Linux virtual computer, enabling full desktop apps plus a browser, and frames the core value as universality: agents can use the same software humans use.

The major leap in Computer Use is reliability through debugging and multi-step execution, not just clicking faster.

He contrasts earlier systems that could start tasks but got stuck, with current agents that retry, introspect, and debug; he also notes agents may write and execute JavaScript to perform many actions at once.

Better representations (App Shots + accessibility/DOM access) reduce “scroll-and-screenshot” loops and speed up automation.

He describes App Shots as capturing raw text and an accessibility representation (not just pixels), making it token-efficient and giving the model richer context like link targets and untruncated fields.

OpenAI is emphasizing permissions and safety patterns as Computer Use moves into the Agents API.

He says developers should build trust via reliability, safety checks, user consent for consequential actions (like payments), and restricting access to only necessary sites/apps.

WORDS WORTH SAVING

5 quotes

Dots are a really cool product because each Dot has access to its own Linux virtual computer in the cloud, uh, which is different from our other products.

— Ari Weinstein

I hope they have a different perspective now because Computer Use is, like, 180 degrees different than it was.

— Ari Weinstein

Now the models just in the last one year have become extraordinarily capable at computer use. I think the biggest delta that I see is before they could like reliably start tasks, but then they would run into problems, and now they're really good at debugging.

— Ari Weinstein

I think what's really crazy that I think, you know, the team's accomplished over the past couple months is that now Computer Use is, like, faster at accomplishing tasks than, like, the average human probably in, in most cases.

— Ari Weinstein

Define our roadmap for us, please.

— Nikunj Handa

Computer Use performance and “superhuman” speedDots with personal Linux cloud computersAgents API integration of Computer UseApp Shots and accessibility/DOM representationsAsync tool calling and mid-turn steering via WebSocketsDecisions API: parallel inference, structured outputs, TTFT optimizationCaching, pre-warming, and context compaction for long-running agents

High quality AI-generated summary created from speaker-labeled transcript.

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.