Lex Fridman PodcastJeremy Howard: fast.ai Deep Learning Courses and Research | Lex Fridman Podcast #35
CHAPTERS
- 0:00 – 1:21
Meet Jeremy Howard and the fast.ai ethos
Lex Fridman introduces Jeremy Howard and frames fast.ai as a practical, accessible entry point into deep learning. The tone is set: focus on real-world results, minimal hype, and learning by doing.
- 1:21 – 3:01
First code: using a Commodore 64 to search for better musical scales
Jeremy recounts his earliest remembered program: a BASIC search over musical scale sizes to find more accurate harmonic ratios. The discussion ties programming curiosity to a lifelong relationship with music.
- 3:01 – 8:23
Programming environments that make data feel easy: Access, Excel, and relational thinking
Jeremy argues that developer productivity often comes from environment and data tooling more than language elegance. He highlights Microsoft Access as an unusually powerful way to rapidly build data-driven apps, and laments how hard modern stacks make similar workflows.
- 8:23 – 12:59
The “third path” in languages: APL lineage, J, and array-oriented programming
Jeremy introduces array-oriented languages as a major (but overlooked) programming family alongside functional and imperative/OO traditions. He explains why J is extraordinarily expressive and how related languages power high-speed finance systems.
- 12:59 – 15:01
Pragmatism wins: Perl, Python, and why libraries dictate reality
Despite love for more elegant languages, Jeremy explains why real projects gravitate to ecosystems with libraries and momentum. He discusses building Fastmail entirely in Perl and why Python ultimately replaced Perl in many domains despite being less elegant.
- 15:01 – 21:42
Making deep learning truly hackable: Swift, MLIR, and DSLs for GPU kernels
Jeremy outlines why Python slows down innovation in core deep learning kernels and argues for an “infinitely hackable” stack. He highlights compiler-based approaches (MLIR, TVM, Halide, tensor DSLs) as the route to high performance without CUDA-level boilerplate.
- 21:42 – 23:34
Hardware reality: NVIDIA lock-in, TPUs, and the need for competition
The conversation turns to the current dependence on CUDA/NVIDIA and why it limits both cost and innovation. Jeremy critiques TPU programmability and describes how backend-agnostic compiler stacks could open the door to alternative accelerators.
- 23:34 – 28:16
From Enlitic to fast.ai: deep learning for medicine and the doctor shortage
Jeremy connects fast.ai’s origin story to his earlier company Enlitic, aimed at deep learning in medicine. He frames the core medical opportunity as amplifying scarce expertise—especially in developing regions—via triage and decision support.
- 28:16 – 32:26
Why medical AI adoption is slow: regulation, hospital lawyers, and data portability
Jeremy describes the non-technical barriers that keep medical AI from scaling, emphasizing institutional interpretation of regulation over regulation itself. The discussion expands into privacy, incentives, and why data sharing often fails despite huge potential upside.
- 32:26 – 37:59
Privacy and “do more with less”: transfer learning as a practical antidote
Jeremy argues many data-hungry narratives are incentive-driven and not strictly necessary. He emphasizes transfer learning’s ability to achieve state-of-the-art results with far less data, and highlights user-controlled medical data as a better model for sharing.
- 37:59 – 40:56
fast.ai’s mission: empower domain experts (who may not code) to use deep learning
Jeremy explains that after seeing deep learning’s broad applicability, he chose to maximize impact by enabling people already embedded in domain problems. fast.ai is framed as a practical upskilling pathway that respects time constraints and real-world needs.
- 40:56 – 45:41
Theory vs practice: why much deep learning research misses what matters
Jeremy criticizes academic incentive structures that reward incremental work on fashionable topics rather than practical breakthroughs. He highlights under-invested areas like transfer learning and active learning, and recounts how a course-driven project led to ULMFiT.
- 45:41 – 52:02
DAWNBench: fast.ai’s speed-and-cost wins and the ImageNet resolution trick
Jeremy tells the story of joining DAWNBench late, rapidly applying best practices, and outperforming major players on CIFAR-10 and later ImageNet. A key insight: train on low-resolution images first, then fine-tune at full resolution to save time while meeting accuracy thresholds.
- 52:02 – 58:57
Single-GPU creativity: smaller benchmarks, DeOldify, and an underused audio frontier
Jeremy argues that multi-machine training often slows iteration and can distort research priorities. He promotes smaller, well-chosen datasets for faster experimentation, discusses GAN-quality results without GANs, and points to audio as an overlooked application area fast.ai wants to support.
- 58:57 – 1:06:16
Learning-rate breakthroughs: superconvergence and the limits of current DL science norms
Jeremy explains Leslie Smith’s “superconvergence” and how fast.ai leveraged it for competitive training speedups. He critiques the field’s resistance to publishing strong empirical results without full theoretical explanations and predicts more automation and fewer tuning dials.
- 1:06:16 – 1:17:51
Cloud, tooling, and frameworks: why fast.ai chose PyTorch (and where Swift fits)
The discussion covers practical compute options (GCP, AWS, one-click notebook platforms) and the teaching impact of lowering setup friction. Jeremy contrasts static-graph frameworks with PyTorch’s dynamic approach, explains fast.ai’s layered API philosophy, and critiques TensorFlow’s technical debt while remaining optimistic about Swift for TensorFlow.
- 1:17:51 – 1:26:59
How to get started (and become an expert): train models, fine-tune, and pick a domain
Jeremy gives concrete advice for learning deep learning: run many experiments, inspect inputs/outputs, and quickly fine-tune models on your own data. He describes fast.ai’s workflow (including scraping datasets and deploying simple apps) and stresses that expertise should be anchored in a domain you care about.
- 1:26:59 – 1:32:21
Startups and independence: tenacity, real problems, and avoiding VC pressure
Jeremy connects startup success to the same trait he sees in learning: not giving up. He advocates for low-cost, revenue-first approaches, warns against commercializing PhD topics that don’t solve real problems, and explains why VC incentives can distort decision-making.
- 1:32:21 – 1:40:05
Learning how to learn: spaced repetition, Anki, and long-horizon consistency
Jeremy explains spaced repetition (Ebbinghaus) and how tools like Anki operationalize memory scheduling. He shares why he uses it primarily for Chinese, emphasizes mnemonics and context, and argues that consistency—and forgiving lapses—is the hardest and most important part.
- 1:40:05 – 1:44:10
What matters next: stop predicting AGI, solve real problems, and take ethics seriously
Jeremy refuses to speculate on human-level AI timelines, arguing there’s no grounding data and the question distracts from urgent, solvable challenges. He focuses on societal impacts like labor displacement and calls on data scientists to treat ethics as a core responsibility in system design and deployment.
