By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

This Startup Beat Gemini 3 on ARC-AGI — at Half the Cost

Poetiq is a new startup founded by former DeepMind researchers that recently achieved a major jump on the ARC-AGI benchmark by layering a recursive self-improvement system on top of Gemini 3.

Francois ChaubaurdhostIan Fisherguest

Jan 29, 202611mWatch on YouTube ↗

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

CHAPTERS

Jump to publisher timestamps

Chapter summaries

Poetiq’s ARC-AGI 2 result: 54% on the private test set
Francois introduces Ian Fisher at NeurIPS and tees up Poetiq’s headline announcement. Ian explains Poetiq’s stack running on top of Gemini 3 achieved 54% on the ARC-AGI 2 private test set, positioning it as a major jump over prior results.
- •Poetiq is a newly formed company (started in June), largely ex-DeepMind
- •Core announcement: 54% on ARC-AGI 2 private test set using Poetiq + Gemini 3
- •Framing the result as a step-change vs previous state of the art
Quantifying the gain: better than Gemini 3 DeepThink at half the cost
The discussion pins down what the improvement means relative to Gemini 3 baselines. Ian emphasizes the fairest comparison is against Gemini 3 DeepThink—Poetiq scores ~9–10 points higher while costing about half as much.
- •Gemini 3 baseline cited around low-30% range (approximate)
- •Gemini 3 DeepThink baseline cited at ~45%
- •Poetiq claims ~54%: ~9–10 points better than DeepThink
- •Cost advantage: Poetiq is about half the cost of DeepThink
Ian Fisher’s background: YC founder → Google acquisition → ML research focus
Ian summarizes his entrepreneurial and research journey that led to Poetiq. He describes founding a prior YC company acquired by Google, then switching into Google Research and refocusing once LLMs became the dominant wave.
- •Poetiq is Ian’s third company; second was YC-backed Affordable
- •Affordable was acquired by Google in 2015
- •Ian transitioned into Google Research for fundamental ML work
- •LLMs motivated a major research refocus that seeded Poetiq
Recursive self-improvement (RSI): the “holy grail,” but must be done safely
Ian outlines Poetiq’s core thesis: faster, cheaper recursive self-improvement where AI systems make themselves better. He acknowledges broad competition and highlights the importance—and debate—around safety and desirability.
- •RSI framed as a central prize in AI: systems that improve themselves
- •Poetiq’s approach is positioned as faster and cheaper than typical paths
- •Competitive landscape includes major labs and startups
- •Safety is emphasized as a key requirement for RSI work
Why ARC (and ARC-AGI 2) became the proving ground—plus the Gemini 3 timing
Francois asks about targeting ARC and the ‘hill-climbing’ experience as Gemini 3 arrived. Ian explains Poetiq was initially focused on ARC 1, with ARC-AGI 2 used more as a sanity check—until Gemini 3 produced a ‘holy cow’ jump.
- •Initial focus was ARC 1; ARC-AGI 2 seemed much harder
- •Poetiq tested across different API model providers
- •Gemini 3 release changed performance dynamics significantly
- •ARC 1 improved from ~89% (other models) to ~95% (Gemini 3)
- •Running on ARC-AGI 2 after Gemini 3 yielded an unexpected surge
RSI loop mechanics without weight access: prompts + surrounding system + evals
Ian explains that Poetiq improves performance by iterating on tasks with measurable evaluation, even without access to model weights. The action space is not fine-tuning—it's prompt and system optimization around black-box API models.
- •RSI loop uses other evaluable tasks to drive improvement
- •No access to weights: optimization focuses on prompts and system design
- •System-level scaffolding is part of the controllable action space
- •Improvement is driven by structured iteration and measurement
Ensembles, self-refinement, and voting: system-level optimization beyond single prompting
Ian describes the solver architecture: multiple independent ensemble members call the base model multiple times to refine answers, then a voting/combination strategy selects the final output. Francois contrasts this with earlier DSPy-style approaches and notes Poetiq seems to achieve materially better results.
- •Ensemble members perform independent multi-step refinement
- •Multiple calls to the underlying model per problem
- •A voting/aggregation scheme combines candidates into a final answer
- •Positioned as similar in spirit to DSPy, but with additional proprietary improvements
Why Gemini 3 “changed everything”: coding + visual problem-solving capability
Ian attributes much of the leap to Gemini 3’s strength at writing code for visual problem solving, which maps well onto ARC-style tasks. He also notes Anthropic’s Opus 4.5 appears comparable in quality but more expensive, and swapping base models can yield similar outcomes.
- •Gemini 3 is described as unusually strong at coding for visual reasoning
- •This capability is linked to better ARC solver performance
- •Opus 4.5 shows similar quality in their tests but at higher cost
- •Poetiq’s approach is model-agnostic enough to swap base providers
What’s next: more high-impact benchmarks, staying small, and building a business
Ian says Poetiq plans to pursue additional important benchmarks (not disclosed) while also engaging with real customers. He notes the team is only six people (with a seventh joining), aiming to pair research momentum with market impact.
- •Plans include tackling more ‘high-impact’ benchmarks (kept confidential)
- •Poetiq team size: six, with a seventh starting soon
- •Transitioning from purely research wins to customer conversations
- •Goal: solve real business problems while advancing RSI capabilities
Is RSI a path to AGI? ‘A bump’ now, but potentially a route to AGI and beyond
Francois challenges whether RSI is merely incremental (like a small regularization boost) or fundamentally transformative. Ian argues both: you always want the performance bump, and RSI is among the most exciting plausible paths to AGI—though not the only one.
- •RSI provides pragmatic performance gains worth pursuing
- •Poetiq leveraged ARC rules (multiple-solution setting) to be cheaper and better in this context
- •In single-response settings, system overhead can make Poetiq costlier or equal-cost
- •Ian views RSI as a credible path to AGI and beyond, but not exclusive
When to stop hill-climbing: compute cost, budget constraints, and prioritizing customers
Francois asks whether performance plateaued or was stopped intentionally. Ian says they stopped hill-climbing because ARC-AGI runs were expensive and they needed to conserve resources for customer work—implying more gains may have been possible with more spend.
- •Hill-climbing on ARC-AGI was expensive to run
- •They stopped due to budget and operational constraints, not necessarily a plateau
- •Tradeoff: pushing benchmarks further vs reserving compute for customers
- •Implicit claim: further improvements may be possible with more funding
Automating prompt engineers and agent builders: from ‘hand-built car’ to ‘car factory’
The conversation closes on product vision: automating the human loop in prompt/agent development. Ian contrasts earlier DeepMind-era manual system-building with Poetiq’s approach—building a ‘factory’ that automates the creation and improvement of these systems.
- •Customer-facing excitement alongside research roadmap
- •Prompting is often the main controllable lever when models improve but remain black-box
- •Poetiq aims to automate prompt engineering and agent construction end-to-end
- •Analogy: moving from hand-assembling a system to building a factory that produces systems

Publisher timestamps

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.