YC Root AccessThis Startup Beat Gemini 3 on ARC-AGI — at Half the Cost
CHAPTERS
- 0:11 – 0:49
Poetiq’s ARC-AGI 2 result: 54% on the private test set
Francois introduces Ian Fisher at NeurIPS and tees up Poetiq’s headline announcement. Ian explains Poetiq’s stack running on top of Gemini 3 achieved 54% on the ARC-AGI 2 private test set, positioning it as a major jump over prior results.
- •Poetiq is a newly formed company (started in June), largely ex-DeepMind
- •Core announcement: 54% on ARC-AGI 2 private test set using Poetiq + Gemini 3
- •Framing the result as a step-change vs previous state of the art
- 0:49 – 1:16
Quantifying the gain: better than Gemini 3 DeepThink at half the cost
The discussion pins down what the improvement means relative to Gemini 3 baselines. Ian emphasizes the fairest comparison is against Gemini 3 DeepThink—Poetiq scores ~9–10 points higher while costing about half as much.
- •Gemini 3 baseline cited around low-30% range (approximate)
- •Gemini 3 DeepThink baseline cited at ~45%
- •Poetiq claims ~54%: ~9–10 points better than DeepThink
- •Cost advantage: Poetiq is about half the cost of DeepThink
- 1:16 – 1:49
Ian Fisher’s background: YC founder → Google acquisition → ML research focus
Ian summarizes his entrepreneurial and research journey that led to Poetiq. He describes founding a prior YC company acquired by Google, then switching into Google Research and refocusing once LLMs became the dominant wave.
- •Poetiq is Ian’s third company; second was YC-backed Affordable
- •Affordable was acquired by Google in 2015
- •Ian transitioned into Google Research for fundamental ML work
- •LLMs motivated a major research refocus that seeded Poetiq
- 1:49 – 2:37
Recursive self-improvement (RSI): the “holy grail,” but must be done safely
Ian outlines Poetiq’s core thesis: faster, cheaper recursive self-improvement where AI systems make themselves better. He acknowledges broad competition and highlights the importance—and debate—around safety and desirability.
- •RSI framed as a central prize in AI: systems that improve themselves
- •Poetiq’s approach is positioned as faster and cheaper than typical paths
- •Competitive landscape includes major labs and startups
- •Safety is emphasized as a key requirement for RSI work
- 2:37 – 3:41
Why ARC (and ARC-AGI 2) became the proving ground—plus the Gemini 3 timing
Francois asks about targeting ARC and the ‘hill-climbing’ experience as Gemini 3 arrived. Ian explains Poetiq was initially focused on ARC 1, with ARC-AGI 2 used more as a sanity check—until Gemini 3 produced a ‘holy cow’ jump.
- •Initial focus was ARC 1; ARC-AGI 2 seemed much harder
- •Poetiq tested across different API model providers
- •Gemini 3 release changed performance dynamics significantly
- •ARC 1 improved from ~89% (other models) to ~95% (Gemini 3)
- •Running on ARC-AGI 2 after Gemini 3 yielded an unexpected surge
- 3:41 – 4:01
RSI loop mechanics without weight access: prompts + surrounding system + evals
Ian explains that Poetiq improves performance by iterating on tasks with measurable evaluation, even without access to model weights. The action space is not fine-tuning—it's prompt and system optimization around black-box API models.
- •RSI loop uses other evaluable tasks to drive improvement
- •No access to weights: optimization focuses on prompts and system design
- •System-level scaffolding is part of the controllable action space
- •Improvement is driven by structured iteration and measurement
- 4:01 – 5:01
Ensembles, self-refinement, and voting: system-level optimization beyond single prompting
Ian describes the solver architecture: multiple independent ensemble members call the base model multiple times to refine answers, then a voting/combination strategy selects the final output. Francois contrasts this with earlier DSPy-style approaches and notes Poetiq seems to achieve materially better results.
- •Ensemble members perform independent multi-step refinement
- •Multiple calls to the underlying model per problem
- •A voting/aggregation scheme combines candidates into a final answer
- •Positioned as similar in spirit to DSPy, but with additional proprietary improvements
- 5:01 – 6:18
Why Gemini 3 “changed everything”: coding + visual problem-solving capability
Ian attributes much of the leap to Gemini 3’s strength at writing code for visual problem solving, which maps well onto ARC-style tasks. He also notes Anthropic’s Opus 4.5 appears comparable in quality but more expensive, and swapping base models can yield similar outcomes.
- •Gemini 3 is described as unusually strong at coding for visual reasoning
- •This capability is linked to better ARC solver performance
- •Opus 4.5 shows similar quality in their tests but at higher cost
- •Poetiq’s approach is model-agnostic enough to swap base providers
- 6:18 – 7:10
What’s next: more high-impact benchmarks, staying small, and building a business
Ian says Poetiq plans to pursue additional important benchmarks (not disclosed) while also engaging with real customers. He notes the team is only six people (with a seventh joining), aiming to pair research momentum with market impact.
- •Plans include tackling more ‘high-impact’ benchmarks (kept confidential)
- •Poetiq team size: six, with a seventh starting soon
- •Transitioning from purely research wins to customer conversations
- •Goal: solve real business problems while advancing RSI capabilities
- 7:10 – 8:39
Is RSI a path to AGI? ‘A bump’ now, but potentially a route to AGI and beyond
Francois challenges whether RSI is merely incremental (like a small regularization boost) or fundamentally transformative. Ian argues both: you always want the performance bump, and RSI is among the most exciting plausible paths to AGI—though not the only one.
- •RSI provides pragmatic performance gains worth pursuing
- •Poetiq leveraged ARC rules (multiple-solution setting) to be cheaper and better in this context
- •In single-response settings, system overhead can make Poetiq costlier or equal-cost
- •Ian views RSI as a credible path to AGI and beyond, but not exclusive
- 8:39 – 9:09
When to stop hill-climbing: compute cost, budget constraints, and prioritizing customers
Francois asks whether performance plateaued or was stopped intentionally. Ian says they stopped hill-climbing because ARC-AGI runs were expensive and they needed to conserve resources for customer work—implying more gains may have been possible with more spend.
- •Hill-climbing on ARC-AGI was expensive to run
- •They stopped due to budget and operational constraints, not necessarily a plateau
- •Tradeoff: pushing benchmarks further vs reserving compute for customers
- •Implicit claim: further improvements may be possible with more funding
- 9:09 – 11:23
Automating prompt engineers and agent builders: from ‘hand-built car’ to ‘car factory’
The conversation closes on product vision: automating the human loop in prompt/agent development. Ian contrasts earlier DeepMind-era manual system-building with Poetiq’s approach—building a ‘factory’ that automates the creation and improvement of these systems.
- •Customer-facing excitement alongside research roadmap
- •Prompting is often the main controllable lever when models improve but remain black-box
- •Poetiq aims to automate prompt engineering and agent construction end-to-end
- •Analogy: moving from hand-assembling a system to building a factory that produces systems
