No PriorsNo Priors Ep. 68 | With Zapier Co-Founder and Head of AI Mike Knoop
CHAPTERS
- 0:00 – 2:00
Why Mike Knoop is launching ARC Prize to accelerate AGI progress
Elad introduces Mike Knoop (Zapier co-founder, Head of AI) and tees up the ARC Prize initiative. Mike explains his long-running interest in AGI, why he stepped back from exec duties to focus on AI, and why he believes AGI progress has stalled.
- •Mike’s background at Zapier and focus on application-layer AI
- •Motivation: measuring “how close are we to AGI?” after chain-of-thought era
- •Claim: perceived AGI progress is overstated; real progress has stalled
- •Concern: frontier labs publish fewer technical details, slowing community learning
- 2:00 – 3:08
ARC Prize and the ARC-AGI evaluation: what it is and why it matters
Mike introduces the ARC Prize as a nonprofit, public challenge to beat François Chollet’s ARC-AGI evaluation and open-source the solution. He describes ARC as a rare evaluation that resists brute-force scaling and notes slow progress over multiple years.
- •ARC Prize: $1M+ challenge, open-source requirement
- •ARC-AGI eval described as a strong, principled AGI test
- •Historical results: ~20% to ~34% over four years—limited gains
- •ARC puzzles resemble compact “IQ-test-like” tasks that resist LLM scaling
- 3:08 – 5:12
Competing AGI definitions: economic usefulness vs. fast skill acquisition
Elad asks what the prevailing AGI definition is and what’s wrong with it. Mike contrasts the popular “economically useful work” definition with Chollet’s definition: general intelligence as the ability to efficiently learn new skills from limited experience.
- •Industry consensus: AGI = system that can do most economically useful human work
- •Mike: that definition tracks usefulness, not general intelligence
- •Chollet’s definition: efficiently acquire new skills and solve novel problems
- •Examples: narrow AIs can excel at single domains but can’t transfer skills
- 5:12 – 7:40
Why LLMs generalize yet still fall short of ‘true’ general intelligence
Elad probes why broad, general-purpose LLMs don’t count as sufficient AGI progress. Mike frames LLMs as powerful pattern/memorization systems that struggle with genuinely novel discovery where the answer and reasoning chains aren’t present in training data.
- •LLMs: high-dimensional pattern learners; “memorization” is not full general intelligence
- •Discovery tasks (new physics, therapeutics) require inventing beyond training distributions
- •Agents work when reasoning chains are short/common enough to be abstracted from data
- •Empirical claim: LLM-based attempts have not succeeded on ARC-AGI
- 7:40 – 8:32
Beyond next-token prediction: what ‘more’ is needed for AGI
Mike argues that simply scaling next-token-prediction language models won’t reach AGI. He suggests transformers may still contribute—especially as a perception stack—but that additional mechanisms/architectures are required.
- •Scaling alone (more params/data) won’t solve AGI per Mike’s view
- •Transformers’ key contribution: robust multimodal perception representations
- •Next-token prediction as a sole objective is insufficient
- •Need additional components beyond standard LM training regimes
- 8:32 – 10:50
Promising research directions: program synthesis and architecture search
Elad asks what ideas are missing and what should be explored. Mike highlights program synthesis as a promising approach on ARC, and revisiting neural architecture search now that compute is cheaper and more available.
- •Program synthesis: search through program space to map inputs to outputs
- •Orthogonal to standard LM approaches; has driven some ARC gains
- •Neural architecture search: computers discovering architectures rather than humans
- •Revisiting NAS with modern compute and fewer human priors (‘bitter lesson’)
- 10:50 – 13:48
AGI vs sentience: focusing on capabilities and an empirical lens
Elad asks whether Mike’s AGI concept includes sentience or consciousness. Mike avoids philosophical claims and emphasizes capability-based goals: systems that can learn new skills and help humans solve open-ended problems, evaluated empirically.
- •Distinction between intelligence and sentience raised (risk implications)
- •Mike’s focus: tools that invent/discover alongside humans
- •Avoids claims about consciousness; prioritizes observable capabilities
- •Stresses empiricism over speculation about future systems
- 13:48 – 16:22
Why a prize model (not startups) and why outsiders may win
Elad asks why ARC Prize is structured as a prize rather than investment. Mike argues outsiders are critical, the problem is compact enough to be tackled by small teams, and prizes can redirect talent away from purely commercial LM opportunities.
- •Many ARC competitors are small, globally distributed teams
- •Bet: breakthrough likely from someone not steeped in scale/LM orthodoxy
- •ARC’s compact tasks make solutions feasible without massive models
- •Prize creates status + financial incentive to pursue noncommercial research
- 16:22 – 17:46
Zapier’s AI journey: early internal experimentation and tool use
Elad pivots to Zapier’s AI product journey and how they chose what to build. Mike describes a six-month period of intensive prototyping (chain/tree-of-thought, internal ChatGPT-like systems) leading to a key insight: models needed tools, which matched Zapier’s integration layer.
- •In 2022, Mike and Zapier’s CTO shifted to hands-on IC work to explore capabilities
- •Built internal prototypes including chain/tree-of-thought and a ChatGPT-like experience
- •Identified “frozen in time” limitation pre-tool-use era
- •Hooked LMs to Zapier’s 6,000+ integrations; became a ChatGPT plugin launch partner
- 17:46 – 19:08
Adoption signals and new products: 50M AI tasks and Zapier Central bots
Elad asks about usage metrics and adoption. Mike shares scale (50M+ AI tasks) and explains how users mainly embed AI steps inside workflows; he also introduces Zapier Central, where bots are programmed in natural language via inference rather than rigid mapping.
- •50M+ AI tasks executed on Zapier over ~18 months
- •Dominant use cases: content generation, extraction, summarization within workflows
- •Zapier Central: AI bots that reduce manual configuration and are easier to use
- •Natural-language programming via inference (not just NL-to-structured translation)
- 19:08 – 21:46
Economic value of agents: reliability, risk, and ‘clamping’ behavior
Elad asks how close we are to an agentic future. Mike argues it’s already here in paid usage, and frames progress as expanding “concentric rings” of higher-stakes use cases as reliability improves—enabled by product controls that constrain agent behavior.
- •Agentic automation is already monetized in practice
- •Adoption starts with low-risk personal/team workflows
- •Same bot can be acceptable for startups but too risky for large enterprises
- •Key product challenge: adding clamps/constraints to raise reliability and reduce downside
- 21:46 – 23:38
Open source as the engine for breakthroughs—and why publishing is closing down
Elad asks about open source in AI amid regulatory shifts. Mike argues open research is essential because AGI needs fundamental breakthroughs, and closed frontier research slows idea generation; ARC Prize aims to counterbalance this by requiring open sourcing.
- •AGI progress needs new ideas; open source accelerates idea diffusion
- •Open sharing helps would-be researchers participate without gatekeeping by big labs
- •Commercialization has reduced frontier publication (tight-to-chest behavior)
- •ARC Prize: open-source solutions and open progress to rebuild shared learning
- 23:38 – 26:00
Regulating AI and AGI: use existing frameworks, avoid speculative prohibitions
Elad draws parallels to internet protocols and 1990s cryptography regulation. Mike argues today’s AI harms can be handled by existing regulatory agencies, and that prescriptive AGI legislation based on theoretical predictions is dangerous; policy should follow empirical evidence of capabilities.
- •Analogy to crypto regulation: fears vs eventual net-positive outcomes
- •For current narrow AI, existing regulatory frameworks may suffice
- •Against prescriptive AGI laws before evidence of actual capabilities
- •Values personal freedom; prefers capability-informed decisions on release/control
