At a glance
WHAT IT’S REALLY ABOUT
Why unreliable AI web agents fail—and how to build better
- Most “do anything on the web” agents fail because small per-step error rates compound into very low end-to-end success on long, multi-step workflows.
- Das argues the industry is dangerously normalizing non-determinism and “slop,” and that agents should work on the first try to be considered shippable.
- Yutori focuses on robustness via comprehensive production evals, guardrails, and the ability for agents to recognize mistakes, backtrack, and recover.
- The product differentiator is shifting from quick prototypes to taste and craft, developed through rigorous internal dogfooding and an 80/20 focus on high-leverage features.
- Trust is built through reliability and “proof of work” transparency—showing users what the agent did (sites visited, steps taken)—a philosophy connected to Das’s interpretability background (Grad-CAM).
IDEAS WORTH REMEMBERING
5 ideasLong workflows magnify small agent errors into frequent failures.
If an agent is 90% correct per step, a 10–50 step workflow quickly collapses in end-to-end success, which is why many impressive demos don’t hold up in real use.
“Usually works” is a bad standard for agentic products.
Das argues reliability shouldn’t be negotiated downward; if an agent can’t succeed on the first attempt most of the time, shipping it as a general solution erodes user trust and product credibility.
Recovery behavior matters as much as raw task competence.
Because agents will encounter unfamiliar websites and layouts, the critical capability is noticing mistakes and backtracking to try a different path, not pretending errors won’t happen.
Continuous evals in production are essential for improving agents.
Yutori runs each production query through comprehensive evaluations to pinpoint where agents succeed or fail and to prioritize which domains and behaviors need more work.
Web-agent generalization will always be incomplete—design for it.
No team can train on “every website,” so systems must expect novelty and variance and rely on guardrails, error detection, and correction loops to stay dependable.
WORDS WORTH SAVING
5 quotesIn this day and age, there's basically 100 different agent products out there that are saying that, like, this can do anything on the web, and you try it once, and it doesn't really work.
— Abhishek Das
If we think of a 10-step, 20-step, or 50-step workflow, even if the accuracy at each step is, like, 90%, the 10% error rate compounds very quickly, and so the overall success rate of a task, of a workflow is, like, quite low, right?
— Abhishek Das
Yeah, if it's not good enough to work on the first try, it's not good enough.
— Abhishek Das
In a world where it's very easy to come up with first prototypes using these coding LLMs, the true differentiator is in taste and craft in how intuitive and well-designed the product is.
— Abhishek Das
Whenever we ship something, we have to get it right. It has to really work. It has to be reliable. Users have to trust that it works well.
— Abhishek Das
High quality AI-generated summary created from speaker-labeled transcript.
