Evaluating the Fine-Grained Planning Abilities of Web Agents

At our inaugural YCML at Startup School, YC Partner Ankit Gupta speaks with Surgan Jandial about evaluating how web agents plan, rather than only measuring whether they complete a task. The work breaks planning into individual skills such as temporal ordering, future-state prediction, action selection, and error correction. It then repurposes existing datasets to create inexpensive synthetic tests that measure each skill independently. These scores correlate with agents' performance on complete web tasks, providing a cheaper and more interpretable way to identify their weaknesses before running full trajectory evaluations. Apply to Y Combinator: https://www.ycombinator.com/apply Work at a startup: https://www.ycombinator.com/jobs

Ankit GuptahostSurgan Jandialguest

Aug 6, 20266mWatch on YouTube ↗

EPISODE INFO

Released: August 6, 2026
Duration: 6m
Channel: YC Root Access
Watch on YouTube: ▶ Open ↗

EPISODE DESCRIPTION

At our inaugural YCML at Startup School, YC Partner Ankit Gupta speaks with Surgan Jandial about evaluating how web agents plan, rather than only measuring whether they complete a task. The work breaks planning into individual skills such as temporal ordering, future-state prediction, action selection, and error correction. It then repurposes existing datasets to create inexpensive synthetic tests that measure each skill independently. These scores correlate with agents' performance on complete web tasks, providing a cheaper and more interpretable way to identify their weaknesses before running full trajectory evaluations. Apply to Y Combinator: https://www.ycombinator.com/apply Work at a startup: https://www.ycombinator.com/jobs

SPEAKERS

Ankit Gupta
host
Host on YC Root Access (Y Combinator), interviewing researchers about their work in machine learning.
Surgan Jandial
guest
Master’s student at CMU and researcher presenting work on evaluating fine-grained planning abilities of VLM web agents (EMNLP).

EPISODE SUMMARY

In this episode of YC Root Access, featuring Ankit Gupta and Surgan Jandial, Evaluating the Fine-Grained Planning Abilities of Web Agents explores testing web-agent planning via cheap, interpretable skill-based evaluations framework Web agents repeatedly generate a plan/thought from UI context and then choose the next interaction, but evaluating the quality of that intermediate planning step is hard and expensive compared to scoring final outputs.

RELATED EPISODES