No PriorsNo Priors Ep. 31 | With Cerebras CEO Andrew Feldman
CHAPTERS
- 0:06 – 0:47
GPU crunch and Cerebras’ bet on wafer-scale AI supercomputers
The hosts frame the episode around the compute shortage affecting AI companies and introduce Cerebras Systems as an alternative to GPU-based scaling. Andrew Feldman joins to discuss Cerebras’ approach to building extremely large, purpose-built AI processors and systems.
- •GPU/compute scarcity is delaying training runs and product launches
- •Cerebras positioned as a “rescue” option via purpose-built AI hardware
- •Introduction of CEO Andrew Feldman and his semiconductor startup background (SeaMicro, AMD acquisition)
- 0:47 – 1:41
The G42 partnership: building nine 4-exaflop AI supercomputers
Feldman explains the strategic partnership with G42 and the scale of the deployment: nine supercomputers totaling 36 exaflops of AI compute. He emphasizes the rarity of finding a partner aligned on building at the extreme end of compute scale.
- •Strategic partnership announcement with G42
- •Nine supercomputers, each ~4 exaflops (36 exaflops total)
- •Shared vision and ambition to build “the biggest computer on earth”
- 1:41 – 2:41
Origin story (2015–2016): questioning GPUs and committing to “vastly better” hardware
Feldman recounts how the founding team started meeting in late 2015 and challenged the assumption that GPUs were ideal for AI. They concluded they could build something dramatically better and chose wafer-scale technology to address looming scaling constraints.
- •Catalyst question: why a pixel-pushing GPU should be ideal for AI
- •Decision to aim for a step-change improvement, not incremental gains
- •Early belief that AI workloads would demand massive scale and bandwidth
- •Choosing wafer-scale despite it “never working before”
- 2:41 – 4:27
Wafer-scale rationale: dinner-plate chip to reduce bandwidth and distribution pain
Cerebras’ chip is described as wafer-scale—physically much larger than conventional chips—to mitigate memory bandwidth limits and the complexity of distributing work across many small devices. Feldman notes they underestimated the eventual market size but got the architectural direction right even before transformers.
- •Wafer-scale chip roughly “dinner plate” sized vs postage-stamp chips
- •Designed to avoid memory bandwidth bottlenecks and excessive sharding
- •Anticipated the difficulty of spreading AI workloads over many machines
- •Architecture predates transformers yet performs strongly on them
- 4:27 – 6:07
How Cerebras measures performance: customer time-to-train, not canned benchmarks
Rather than focusing on standardized benchmarks, Feldman argues the only meaningful metric is how long it takes customers to train real models across a suite of sizes. He highlights the hidden cost of distributed training work and teams needed to make GPU clusters run efficiently.
- •Skepticism of canned benchmarks due to “benchmark gaming”
- •Real metric: end-to-end time from customer model to converged run
- •Typical workflow spans many model sizes during tuning (100M→1B→13B+)
- •Distributed training on GPUs can require months and large specialist teams
- 6:07 – 8:16
Inside the Cerebras architecture: dataflow, 850k tiles, and linear scaling via data parallelism
Feldman details the dataflow design: ~850,000 identical compute+memory tiles with local SRAM and token-driven execution. The system streams parameters from an external store and enables strictly data-parallel training that can scale linearly across nodes.
- •Dataflow architecture with many identical tiles (compute + memory)
- •Fully distributed memory: high bandwidth due to proximity (one cycle away)
- •Large on-wafer SRAM avoids typical bandwidth constraints and helps sparsity
- •Parameters streamed from a perimeter store; compute/memory roles separated
- •Strictly data-parallel scaling across clusters (same weights, different data)
- 8:16 – 9:47
Why GPUs struggle at scale: fixed HBM and tightly coupled memory-to-compute ratios
The conversation contrasts GPU constraints (fixed memory configurations and coupling of compute with memory) with Cerebras’ disaggregated approach. Feldman argues this unlocks flexibility for very large models and easier debugging without forcing customers to buy unwanted resources.
- •GPU memory is fixed (e.g., 40GB/80GB), limiting configuration flexibility
- •To get more memory on GPUs, you often must buy more compute too (and vice versa)
- •Disaggregating parameter storage from compute enables custom ratios
- •Ability to run and inspect extremely large models (even trillion-parameter) on one system for debugging and iteration
- 9:47 – 11:44
First workloads and open-source releases: BTLM, Arabic LLM, and “proof by training”
Feldman describes early applications with G42 and why Cerebras began training and releasing open-source models: to demonstrate ease of use and scaling. He cites BTLM’s popularity and an upcoming open-source release of a large Arabic model with partners.
- •BTLM: 3B-parameter model with strong adoption (Hugging Face downloads)
- •Fortune 100 usage and vertical interest (healthcare and beyond)
- •Open-sourcing a large Arabic LLM with Inception and MBZUAI
- •Motivation: show the world training is simpler on Cerebras systems
- •Ability to scale/replicate runs easily compared to GPU cluster redistribution
- 11:44 – 12:17
Model size vs serving economics: why many production models stay smaller
Feldman explains that while very large models are impressive, serving costs rise sharply, pushing many real deployments to 13B parameters or smaller. Cerebras continues to train models across a range, but the economic reality of inference shapes what reaches production.
- •Large models are expensive to train and especially expensive to serve
- •Serving cost scales steeply with parameter count (illustrative comparison)
- •Many production systems favor smaller models (≈13B and below)
- •Cerebras training efforts still span mid-to-large sizes (30B–175B)
- 12:17 – 15:30
Multilingual and cultural representation: data scarcity, tokenization, and norms
The group discusses the coming wave of multilingual and region-specific models and the challenges in building them due to English-dominated corpora. Feldman highlights how tokenization and RLHF/annotation choices can bias models, making careful design essential for fair cultural representation.
- •Underrepresented languages face token/data scarcity due to English-heavy web scrapes
- •Nations want language and culture reflected accurately in LLMs
- •Cultural norms can be encoded via data curation and human feedback processes
- •Tokenization can introduce bias, especially for scripts requiring different encodings
- •Example focus on Arabic/Hebrew/Hindi and the need for thoughtful token schemes
- 15:30 – 17:49
What it takes to start a chip company: long cycles, high burn, and heavy QA
Feldman outlines the structural difficulty of chip startups: long development timelines, large upfront cost, and delayed customer feedback. He explains why verification/QA dominates headcount and why chip businesses still require extensive software stacks to be usable by ML practitioners.
- •Prototype-to-feedback loop is years and tens of millions of dollars
- •Engineering management differs from software (can’t run endless short sprints)
- •Verification/QA is critical because a major bug can force an expensive respin
- •System companies are heavily software-driven (compiler stack, PyTorch compatibility)
- •Chip as “muscle,” software as “brain,” requiring multiple engineering disciplines
- 17:49 – 19:45
AI accelerator market dynamics: NVIDIA dominance, buyer segments, and switching levers
Feldman argues NVIDIA executed extremely well but is now constrained by price and supply, creating opportunities for alternatives. He describes who buys accelerators (enterprises, startups, government) and how “try before you buy” plus deployment help can drive adoption.
- •NVIDIA strength acknowledged, but high prices and limited supply create openings
- •Customers dislike single-vendor dependency (historical parallels to Intel reliance)
- •Key buyer groups: enterprises, generative AI startups, military/intelligence, supercomputing
- •Case study: dramatic training-time reduction leading to customer conversion
- •Use of a Cerebras cloud offering to let customers start training immediately
- 19:45 – 23:27
Specialization vs generality: future silicon for inference, training, and model types
The discussion turns to whether chips will fragment by workload (training vs inference) or by model family (transformers, diffusion, etc.). Feldman emphasizes the cost of specialization and argues Cerebras targets the underlying linear algebra and sparsity rather than betting on specific layer types, while expecting divergence between training and inference hardware.
- •Core architecture trade-off: what to specialize vs keep general
- •Specialized circuits impose opportunity cost (idle silicon, power, area)
- •Cerebras focuses on sparse linear algebra beneath specific model architectures
- •Expectation: training vs inference silicon will diverge, especially for generative inference
- •Generative inference requires holding more state and remains very expensive today
- 23:27 – 26:09
Why the compute supply crunch persists: fab inflexibility, forecasting errors, and long lead times
Feldman explains the GPU crunch as a supply-chain forecasting and capacity-allocation problem, not a simple demand spike. Fabs like TSMC can’t rapidly reallocate capacity, and mistaken forecasts lock companies into long delays or costly over-ordering, implying continued volatility.
- •Semiconductor supply chain is inflexible; capacity changes take many months
- •Forecast misses (including NVIDIA’s) compound allocation constraints at foundries
- •Wafers are pre-allocated across major customers (AMD, Qualcomm, etc.)
- •Over-ordering is also risky because commitments are hard to unwind
- •Expectation that industry will keep underestimating demand for some time
- 26:09 – 30:03
Limits to scaling and the rise of fine-tuning + proprietary enterprise data
Feldman frames scaling as an optimization across accuracy, retraining agility, and inference cost, varying by use case (e.g., radiology vs support bots). He predicts fine-tuning will matter greatly and argues unique enterprise datasets (finance, pharma, news) become strategic assets that may counter the idea that only a few labs will train models.
- •Scaling trade-offs: size vs accuracy, size vs serving cost, size vs update speed
- •Different applications justify different model sizes based on cost of errors
- •Fine-tuning and human feedback can significantly improve model usefulness
- •Unclear equilibrium between training-from-scratch, continuous training, and fine-tuning
- •Unique proprietary datasets (Reuters/Bloomberg/pharma) become “new gold” for differentiation
