CHAPTERS
- 0:18 – 0:49
Base44’s hypergrowth story and the two scaling phases
Yoav frames the talk: how Base44 went from a solo founder to a large engineering organization while keeping speed high using Claude Code. He outlines two key phases—early scaling to ~15 engineers and later scaling to ~80.
- •Talk structure: intro + two growth phases
- •Goal: scale headcount without losing product/engineering velocity
- •Claude Code as an enabler for onboarding, reviews, and quality
- •Phase split: 1→15 engineers, then ~40→80 engineers
- 0:49 – 1:49
Base44 origins: vibe coding platform, traction, and profitability
Yoav introduces Base44 as a “vibe coding” platform aimed at letting anyone build software. He describes rapid product development, building in public, and reaching profitability quickly—setting up the need to scale.
- •Mission: enable both technical and non-technical users to build software
- •Founder built initial product rapidly and publicly (LinkedIn/Twitter)
- •Fast traction led to profitability by April 2025
- •Business momentum triggered the next scaling step
- 1:49 – 2:20
Post-acquisition reality: scaling fast after Wix joins
After Wix acquired/partnered with Base44, the team needed to expand rapidly while preserving startup speed. Yoav describes the immediate jump to ~15 engineers and the pressures that created.
- •Wix alignment: similar user base and big strategic bet
- •Mandate: keep Base44’s velocity while expanding dramatically
- •Team growth: from tiny core to ~15 engineers quickly
- •New organizational scaling challenges surfaced immediately
- 2:20 – 3:22
The four early scaling pain points (onboarding, reviews, customer insight, surface area)
Yoav details why common early-team practices broke at ~15 engineers. The founder couldn’t personally onboard, review every PR, or sit with users, and the product had too many areas for slow ramp-up.
- •Founder-led onboarding didn’t scale
- •Founder-only code review became a bottleneck
- •Manual customer/beta-tester sessions couldn’t cover growth
- •Large product surface required faster ramp-up across domains
- 3:22 – 4:53
Keeping onboarding simple with two Claude prompts (org map + component diagrams)
Instead of maintaining brittle documentation, Base44 used Claude prompts to generate up-to-date context. New engineers used prompts to infer team priorities from commit history and to produce Mermaid diagrams for components.
- •Principle: avoid complex processes; keep it simple
- •Prompt 1: summarize commits to infer what each person/team cares about
- •Prompt 2: generate Mermaid diagrams to understand component behavior
- •Real-time understanding replaces constantly-updated onboarding docs
- 4:53 – 5:24
Scaling code review: distilling founder feedback into repeatable PR guidance
To preserve code quality without founder burnout, they used historical PR comments to extract review principles. Claude helped summarize the “rules” and turn them into lightweight, repeatable instructions for PR review.
- •Founder caution around backend/agent code required consistent standards
- •Use accumulated PR comments as training signal for review guidance
- •Claude summarizes recurring issues and critical considerations
- •Lightweight process repeated every few days, avoiding heavyweight bureaucracy
- 5:24 – 6:54
Velocity proof point: WhatsApp integration shipped in days, not weeks
Yoav shares a concrete example showing the compounding effect of fast onboarding and scalable review. A new engineer delivered a complex WhatsApp integration—from ramp-up to PR to near-production—over a single weekend.
- •Integration spanned multiple domains (APIs, agent flow, Meta API)
- •Expected timeline: 1–2 weeks; actual: Thursday to Sunday
- •Prompts enabled fast context acquisition and execution
- •Claude-assisted review reduced feedback to a few small comments
- 6:54 – 9:43
Production quality without heavy evals: using “frustration” as a simple signal
Rather than building a full eval suite too early, the team mined production traffic for quality signals. They found user frustration in chat correlates strongly with failures, then used Claude to classify frustration and guide rollouts.
- •Full eval suites are high-effort; early teams may not be ready
- •Observation: when things work, users stay quiet; failures create loud complaints
- •Claude classifies user messages by frustration level
- •Use small-percent rollouts to compare frustration across agent versions
- 9:43 – 11:15
Transition to the 40→80 jump: experimentation, evals, and QA become mandatory
Gabriel takes over and explains the next growth shock: headcount doubling overnight after external hiring, internal transfers, and a product merge. This scale introduced new needs: consistent experimentation practices, stronger evals, and scalable QA.
- •Headcount doubled from ~40 to ~80 rapidly
- •New hires can’t rely on implicit tribal knowledge
- •Three focus areas: experimentation at scale, evals, QA without linear tester growth
- •Shift-left decisions and automation become essential
- 11:15 – 12:16
Experimentation at scale: PR-driven guidance for ship vs rollout vs A/B test
They designed an automated workflow that advises engineers on whether to ship directly, do a gradual rollout, or run an A/B test—and for how long. The hard part wasn’t tooling; it was capturing the team’s implicit product judgment into guidelines.
- •Goal: make experimentation decisions consistent and scalable
- •PR-ready trigger envisions a bot commenting with verdicts and KPIs
- •Need to codify intuition: when to test, which metrics, and duration
- •Avoid committees/meetings by deriving rules from historical actions
- 12:16 – 13:47
Turning past behavior into “guidelines”: Claude + PostHog MCP to distill experimentation rules
Instead of writing policies from scratch, they used the last ~100 experiments and corresponding PRs to infer standards. Claude Code connected via PostHog MCP produced an initial guidelines draft that the team iterated quickly.
- •Use historical experiments + PR context as the source of truth
- •Claude Code + PostHog MCP generates first draft of experimentation playbook
- •Guidelines cover KPIs, test duration (days vs weeks), and risk level
- •Outcome: consistent, fast decisions without adding process overhead
- 13:47 – 14:17
Central experimentation command center: dogfooding Base44 connected to BigQuery/PostHog/GitHub
They built a unified dashboard in Base44 to track all experiments, outcomes, and side effects (including AI costs). This made experimentation status visible to everyone and reduced coordination costs across a much larger org.
- •Dogfood Base44 to build internal tooling
- •Integrations: BigQuery (warehouse), PostHog (experiments), GitHub (PR context)
- •Monitor business + product metrics and AI cost side effects
- •Single source of truth for what’s running and what’s moving the needle
- 14:17 – 15:48
Evals become worthwhile: simulate real users and measure latency, turns, and cost
At ~80 people, evals became necessary, but they optimized for fast, practical value rather than a long research project. They built a user simulator that evaluates the system end-to-end by iterating like a real user when something is wrong.
- •Key question: evaluate model output vs correctness of built apps
- •Epiphany: partial failures should trigger repair loops, not just fail the eval
- •User simulator drives agent to fix missing parts, then scores behavior
- •Metrics include latency, number of turns, internal cost, and user credits
- 15:48 – 17:19
CI/CD eval pipeline: spin up real app instances and automate checks with Stagehand
They implemented a pipeline where changes to AI code create real Base44 app instances and run automated user actions. Simple smoke tests (e.g., Hello World) catch breakages, while more complex scenarios validate deeper behaviors like compaction.
- •Every AI code change can trigger end-to-end evaluation
- •Stagehand simulates real user actions in a real app instance
- •Smoke tests (Hello World) verify nothing fundamental broke
- •Advanced scenarios cover iterative edits and complex compaction behavior
- 17:19 – 21:24
Scaling QA with Claude Code: reusable “skills,” fast setup via CLI tools, and PR-driven test plans
To avoid growing manual QA linearly, they operationalized Claude Code as a QA agent. They wrapped common flows into reusable skills, added CLI tools to set up edge-case states quickly, and automated PR-based test planning and reporting (with screenshots and gaps).
- •Problem: deep edge-case testing is tedious and slows feedback loops
- •Create skills for common user flows so Claude doesn’t relearn selectors/paths
- •Add CLI tools to manipulate DB/API state for fast test setup
- •PR triggers: generate test plan, execute, report results + screenshots + limitations
- 21:24 – 23:58
Closing principles and the next bottleneck: simplicity, encoded taste, dogfooding, post-validation
Gabriel summarizes the shared philosophy: do the simplest thing that works, codify “taste” from past actions, and leverage dogfooding for tight feedback loops. He closes with the next challenge—automating post-validation to ensure shipped changes actually achieve intended outcomes.
- •Operate with “bold and simple”; delay complexity until timing is right
- •Encode company taste from historical actions (reviews, experiments, etc.)
- •Dogfooding accelerates learning loops and internal tooling quality
- •Next frontier: automated post-validation after production release
