a16zThe Future of Software Development - Vibe Coding, Prompt Engineering & AI Assistants
CHAPTERS
- 0:00 – 0:48
Why infrastructure matters: the stack keeps getting layered
The conversation opens with the idea that infrastructure doesn’t disappear—it accumulates in layers as new primitives arrive. The panel frames the current moment as unusually important because AI is changing how software itself is built and understood.
- •Infrastructure persists; new capabilities layer on top of old ones
- •AI is disrupting software ("software eating itself")
- •A new infra layer changes how people program and what the stack looks like
- •Developer attention and distribution become central in the AI era
- 0:48 – 2:27
What counts as “infrastructure”: technical buyers, not vertical end-users
They define infrastructure as “the stuff you use to build the stuff,” distinguished primarily by who buys and uses it. Infra is sold to technical users (developers, admins, data scientists), unlike vertical SaaS sold to industry-specific operators.
- •Infra is categorized by technical buyer/user (developer, IT, admin, data scientist)
- •Core components: compute, networking, storage—plus the tooling around development/operations
- •Enterprise apps can be vertical; infra is typically horizontal and cross-industry
- •Go-to-market and diligence differ sharply between technical-buyer infra and vertical SaaS
- 2:27 – 6:34
AI models as the “fourth pillar” of infrastructure
They explore whether models belong alongside compute/networking/storage as a foundational infrastructure layer. Models both depend on the classic pillars and impose new requirements (chips, data centers, latency), while also acting as an intelligence primitive used by many applications.
- •Models rely on and reshape compute, storage, networking requirements
- •Models may be analogous to databases/compute, but are fundamentally new
- •Latency, data production, and training/inference workloads drive new infra needs
- •Models become a pervasive, reusable capability across the software ecosystem
- 6:34 – 17:46
How AI changes the programming model: “abdicating logic” to systems
Martin argues the key novelty is that software is delegating parts of application logic to models, not just resources. This forces a rethink of what “programming” means when outputs are probabilistic and models sometimes “don’t listen.”
- •Past abstraction: resources (compute/storage); new abstraction: logic/answers
- •Non-determinism and reliability concerns reshape engineering practices
- •Analogies (database/network) help, but AI requires a blank-sheet approach
- •This shift is foundational to computer science and software design
- 17:46 – 21:27
Supercycle dynamics: TAM expansion, new behaviors, and startup white space
They compare AI to prior platform shifts (internet, microchip): lowering marginal costs expands markets and creates new user behaviors incumbents struggle to serve. Those new behaviors open room for challengers and new categories of companies.
- •Lower marginal costs expand TAM and attract new users
- •New behaviors create white space for startups versus incumbents’ old motions
- •AI’s scale and novelty mirror the internet-era behavioral shifts
- •Tools enable rapid prototyping and creation from “good ideas” regardless of background
- 21:27 – 22:11
From low-code to natural language: developer tools and the post-COVID acceleration
Jennifer reframes the low-code promise as arriving via natural language and AI assistants. They also highlight COVID as an accelerant for bottom-up adoption and product-led dev tooling—setting the stage for today’s AI dev tools wave.
- •Low-code’s next form is natural language as a programming interface
- •AI tools act as “thought partners” for prototyping and building quickly
- •COVID accelerated PLG/bottom-up adoption of developer tools
- •Developer tool ecosystems flourished as more people tried tools at home
- 22:11 – 25:28
Infra’s evolution at a16z: pre-cloud → cloud → AI, plus the COVID blip
They trace major infra inflection points since a16z’s early days: pre-cloud on-prem software, the cloud transition (new deployment and business metrics), and the current AI transformation. COVID was a distinct, force-majeure shift that changed sales and adoption dynamics.
- •Pre-cloud: on-prem deployments, perpetual licenses, different economics
- •Cloud shift: recurring revenue, NDR/retention, gross margins, new operations
- •AI is described as the most dramatic shift in decades
- •COVID changed enterprise selling and accelerated bottom-up tool adoption
- 25:28 – 27:09
Today’s infra map: dev tools, core infra, and modern data systems
The panel outlines key infra categories they track and invest in: developer tools, compute/network/storage, and data systems. They note the data landscape’s two branches—backend data engineering and analyst-oriented platforms—and why these remain strategically important even amid AI hype.
- •Developer tools: from GitHub to newer AI-native tools like Cursor (mentioned)
- •Core infra remains foundational: compute, networking, storage
- •Data systems split: backend big-data engines vs analyst/tabular ecosystems
- •Examples referenced: Databricks, Fivetran, dbt, Hex, Tabular (acquired)
- 27:09 – 28:32
AI companies blur infra vs apps: why early cycles are hard to classify
In early supercycles, the “new tech becomes the app,” making it difficult to separate infrastructure from applications. They use examples like OpenAI and ElevenLabs to show how model providers often ship both an API platform and an end-user product.
- •Early cycles: infra/app boundaries blur as the market and TAM form
- •Netscape as historical precedent: core tech + product bundled early on
- •OpenAI and ElevenLabs cited as both infra (platform/API) and app (consumer product)
- •“Wrappers” framing fades; building good AI products is now genuinely hard
- 28:32 – 30:59
Defensibility in AI infra: beyond “no moat” and “commoditization”
They revisit earlier skepticism that no layer had defensibility and contrast it with today’s reality: companies across the stack are succeeding simultaneously. The panel argues infra defensibility often comes from hard-to-replicate expertise, integration switching costs, and how stacks consolidate over time.
- •Early view: little defensibility across chips/cloud/models/apps; reality is nuanced
- •Infra moats: deep domain expertise, long build cycles (e.g., databases like DuckDB)
- •Switching costs are high due to embedded integration and logic, even with APIs
- •“Commoditization” is often sloppy; layers tend to retain value and margin
- 30:59 – 34:09
Expansion vs contraction: why zero-sum thinking fails during the boom
Martin describes infra markets as expanding and later contracting into oligopolies/monopolies, without eliminating value in each layer. In expansion phases, aggressive building and investing are rewarded; consolidation later tends to preserve margins rather than destroy them.
- •Expansion phase: market grows; multiple layers can win simultaneously
- •Contraction phase: consolidation leads to oligopoly/monopoly dynamics
- •Margins often persist (e.g., clouds) via scale and tacit price discipline
- •True layer displacement usually requires difficult vertical moves down the stack
- 34:09 – 36:18
Generalization, RL trade-offs, and composing multi-model systems
They debate whether improving frontier models automatically improves every downstream business. Martin questions how well RL-tuned models generalize, while Jennifer argues real-world systems will compose multiple specialized and general models in pipelines rather than rely on a single model for everything.
- •OpenAI’s “model improvements help you” advice depends on generalization behavior
- •RL fine-tuning may introduce trade-offs across capabilities (code vs other tasks)
- •Complex applications often require chaining multiple model calls and steps
- •Future likely includes both frontier general models and task-specialized models
- 36:18 – 40:31
From prompt engineering to context engineering: the new infra opportunity
Reacting to Karpathy-adjacent framing, they argue the core challenge is delivering the right context to models—often requiring classic CS tools (indexes, prioritization) alongside models. This becomes a major new infra frontier: data pipelines, observability, guarantees, and tooling to systematize AI development.
- •“Context engineering” replaces simplistic “prompt engineering” framing
- •Better performance requires curating inputs via indexing, retrieval, prioritization
- •Infrastructure needs: data pipelines, discovery, observability, and guarantees
- •They expect new formalisms and tools for AI software construction within ~5 years
- 40:31 – 43:17
Humans, expectations, and the developer role: more software, more developers
They caution against anthropomorphizing AI (utopia vs doom) and argue professionals remain essential for specification and formalism. Rather than shrinking engineering, they expect more developers and more software—because creation, product decisions, and requirements discovery remain the hard part.
- •AI triggers human projection; they advocate a pragmatic middle path
- •Formal specification still matters; professional disciplines evolve formal systems
- •Likely outcome: more developers empowered by productivity tools, not fewer
- •Value lies in product decisions/workflows/requirements, not just writing code
- 43:17 – 45:16
Synthetic data and agents: what’s real today vs decade-long horizons
They surface ongoing debates: whether synthetic data can meaningfully improve models without new information, and what agents are truly good for now. Coding agents are working well because code offers tight feedback/error correction; general-purpose web agents lag as errors compound in loops.
- •Synthetic data debate: limited gains without new information; skepticism on “self-improving” loops
- •Agent definition: LLMs in a loop; errors can compound without correction
- •Coding agents succeed due to compile/lint/test feedback and structured environments
- •They see agents as a long arc—near-term value is task-bounded, well-specified work
- 45:16 – 47:29
Vertical integration vs horizontal specialization: both will coexist in AI
They argue history shows both vertical and horizontal strategies can win, and AI already exhibits both. OpenAI is positioned as more vertically integrated via ChatGPT, while others emphasize horizontal API distribution; business strategy hinges on where value capture is clearest for a given market and persona.
- •Historical patterns: Apple-style vertical integration vs Microsoft/Intel horizontal specialization
- •AI already shows both: vertically integrated products and horizontal model/API platforms
- •Open models can function as “horizontal” layers because users can’t recreate training
- •Vertical focus requires sharp persona/use-case understanding to capture value
