CHAPTERS
- 0:05 – 2:07
Karan’s path to medical AI: from misinformation projects to safety-first healthcare models
Sarah introduces Karan Singhal and his work leading Med-PaLM 2 at Google. Karan shares how early interest in socially responsible AI (including a “fake news detector” project) led him toward healthcare as a domain where hallucinations, misinformation, and safety matter deeply.
- •Early motivation: using AI for social benefit, initially via misinformation detection
- •Healthcare as a proving ground for safety, factuality, and harm reduction
- •Framing medical QA as a high-stakes alignment problem
- •Karan’s relatively recent shift (last ~1.5 years) into the medical domain
- 2:07 – 4:01
Google research background and the “Brain Moonshot” that created Med-PaLM
Karan explains his earlier work in representation learning and federated/privacy-preserving ML at Google. He then describes how his team pitched and received support through Google Brain’s Moonshot program, which funded the initial Med-PaLM effort with compute and organizational backing.
- •Representation learning and self-supervised learning as foundations for modern LLMs
- •Privacy-preserving ML and federated learning as early focus areas
- •Transition to leading ambitious projects and shaping AI’s trajectory
- •Brain Moonshot as an internal incubator enabling Med-PaLM’s launch
- 4:01 – 6:41
What PaLM is—and how Med-PaLM builds on it
Elad asks for a primer on PaLM and the connection to Med-PaLM. Karan describes PaLM as Google’s Pathways Language Model infrastructure and flagship model, highlighting scaling-driven gains in reasoning, multilinguality, and code that enabled downstream medical capabilities.
- •PaLM: large-scale decoder-only transformer trained via Pathways infrastructure
- •Breakthrough improvements in reasoning, multilingual performance, and coding
- •Minerva and STEM benchmark jumps as evidence of emergent capabilities
- •Motivation to test and adapt these abilities to medical benchmarks
- 6:41 – 8:11
Med-PaLM alignment: why multiple-choice isn’t enough and how instruction prompt tuning helped
The conversation turns to what changed from PaLM/FLAN-PaLM to Med-PaLM. Karan notes strong multiple-choice exam results weren’t matched by safe, high-quality long-form medical answers, motivating targeted alignment using clinician demonstrations and a data-efficient tuning method.
- •FLAN-PaLM crossed key thresholds on USMLE-style MedQA multiple-choice
- •Long-form medical QA exposed safety and quality gaps vs clinician answers
- •Instruction prompt tuning: more than prompting, less costly than full fine-tuning
- •Clinician-provided demonstrations are valuable but expensive to collect
- 8:11 – 9:29
From PaLM to PaLM 2 (and Med-PaLM 2): training objectives, scaling laws, and better data
Karan outlines several contributors to PaLM 2’s improvements that flow into Med-PaLM 2. He emphasizes mixture-of-objectives pretraining, compute/data optimal scaling (Chinchilla-like findings), and upgraded training data—especially multilingual and code.
- •Mixture-of-objectives training (building on UL2 ideas)
- •Using scaling-law insights to balance parameters, compute, and more data
- •Compute-efficient improvements by increasing training data relative to parameters
- •Expanded multilingual and code datasets to broaden model capabilities
- 9:29 – 12:04
A practical framework for domain adaptation: prompting vs prompt-tuning vs fine-tuning vs pretraining
Sarah asks for a decision framework on when to use different adaptation techniques. Karan proposes a data-driven heuristic: few-shot prompting for tiny datasets, prompt-tuning for moderate data when cost matters, and fine-tuning when aiming for top performance with enough examples.
- •Technique choice depends primarily on data quantity and relevance (plus cost/compute)
- •Pretrain from scratch only when you have massive relevant data and resources
- •Prompting for ~3–5 examples; prompt-tuning or light tuning for tens of examples
- •Fine-tuning tends to win when you have ~100+ examples and want best results
- 12:04 – 13:49
Evaluating medical LLMs: gaps in benchmarks and the need for rigorous human review
Karan surveys the prior ecosystem (e.g., Galactica, BioGPT) and explains why evaluation became a central focus for Med-PaLM. He argues that inconsistent, sometimes low-quality benchmarks and insufficient human evaluation made it hard to judge real-world readiness—especially for medical QA.
- •Acknowledges earlier scientific/biomedical LLM efforts and lessons learned
- •Benchmark landscape was fragmented; quality varied across datasets
- •Automated multiple-choice tests are insufficient for real clinical usefulness
- •Detailed human evaluation with physicians and laypeople is essential
- 13:49 – 17:37
Setting the safety bar: counterfactual benefit, real-world workflows, and health-information reality
Elad probes what quality threshold is appropriate given that even clinicians often “Google” in practice. Karan emphasizes balancing high-stakes caution with the reality that people already seek health info online; progress requires workflow-grounded evaluations tied to safety and outcomes.
- •Medical stakes demand strong safety—but status quo information-seeking is imperfect
- •Large share of internet searches are health-related (including by clinicians)
- •Need to evaluate models within specific workflows, not just abstract tasks
- •Focus on both near-term safety and longer-term patient outcomes
- 17:37 – 21:38
Where LLMs fit first: documentation/billing now, higher-stakes physician-assistant workflows next
Elad asks why so many startups focus on drug discovery and where LLMs may land first in healthcare. Karan explains drug discovery has a near-term revenue playbook and existing AI-augmented pipelines, while near-term LLM adoption is likely in documentation/billing and then physician-assistant roles (e.g., radiology support).
- •Drug discovery is popular due to clearer near-term business paths and pipeline insertion points
- •Health-information applications may be higher impact but face greater risk aversion
- •Early deployment: documentation and billing workflows (EHR ecosystem partnerships)
- •Specialized models likely matter more in higher-stakes clinical assistant settings
- 21:38 – 24:54
HIPAA and privacy: redaction, compliance pathways, and the uncertainty of future regulation
The discussion turns to how to architect systems around patient privacy constraints. Karan highlights pragmatic steps like automatic redaction and HIPAA-compliant deployments, while noting that regulation for “software of unknown provenance” remains unsettled and will require serious policy debate as benefits become clearer.
- •Current behavior is messy: people sometimes paste patient data into general LLMs
- •Low-hanging fruit: automated redaction of sensitive identifiers
- •Organizations pursuing HIPAA-compliant approaches, but standards vary
- •Regulatory clarity is evolving; pressure will rise as clinical benefit becomes evident
- 24:54 – 27:46
Privacy-preserving ML and federated learning: ideal vision vs near-term constraints
Sarah asks whether federated learning and privacy-preserving ML will re-emerge as priorities. Karan argues that major near-term advances will likely come from the most capable centralized models (often without needing patient data), but the long-term ideal is user-controlled, privacy-preserving access—possibly via intermediate approaches like trusted execution environments.
- •Med-PaLM models trained without patient health information can still perform surprisingly well
- •Federated learning on health data has high real-world activation energy today
- •Long-term ideal: users control access, can revoke, and still benefit from shared learning
- •Intermediate privacy solutions: trusted execution environments and strong data isolation
- 27:46 – 30:39
Medical assistants as an alignment lab: incentives, high stakes, and scalable oversight challenges
Karan describes why the medical assistant setting may be a better “laboratory” for alignment than general chat. High stakes align incentives to solve safety, and the domain makes scalable oversight concrete—especially as models near physician-level performance and expert evaluation becomes harder.
- •Medical AI forces alignment work due to high stakes and product-release incentives
- •As performance approaches experts, oversight becomes harder and less reliable
- •Scalable oversight becomes acute: distinguishing models vs physicians gets difficult
- •Medical QA provides clear motivation to address hallucinations and value alignment
- 30:39 – 33:45
Beyond RLHF: self-critique, debate, and AI-assisted supervision for evaluation at/above expert level
Elad asks how evaluation works once AI surpasses experts. Karan outlines emerging approaches—self-refinement, debate formats that humans can judge more easily, and AI-mediated supervision via reward models or AI feedback—while acknowledging the field hasn’t solved the problem.
- •Self-critique/self-refinement loops (sometimes guided by human feedback)
- •Debate as a way to reduce verification burden for human judges
- •AI-supervising-AI via reward models; links to RLHF, RLAIF, constitutional AI
- •Open problem: researchers broadly agree scalable oversight isn’t solved yet
- 33:45 – 42:18
Adoption and the next 5 years: raising standard of care, augmenting science, and grounding to authoritative sources
The group closes on adoption dynamics and Karan’s forward-looking view. He expects near-term impact from expanding access and assisting clinicians (telemedicine, decision support, QA for reports) alongside scientific workflow augmentation (biomarkers, literature tools), enabled by multimodality and grounding in authoritative sources with better feedback methods.
- •Medical community reaction: optimism mixed with justified concern and “speed shock”
- •Near-term: broaden access to care, physician assistance, QA/double-checking, telemedicine augmentation
- •Longer-term: scientific copilots—literature search, biomarker discovery, multimodal correlations
- •Key enablers: multimodality, source grounding/attribution, tool use, improved human feedback pipelines
