Silicon Valley GirlStanford Doctor: Never Tell AI What You Think Is Wrong With You
At a glance
WHAT IT’S REALLY ABOUT
Use AI for medical clarity—never feed it your diagnosis guess
- Stanford’s Dr. Jonathan Chen argues that modern AI may outperform many doctors on knowledge-heavy diagnosis tasks, but trust, accountability, and clinical judgment remain essential for real healthcare decisions.
- He explains why people (and even clinicians) can do worse with AI when they ask leading questions, lack domain context, or get anchored—causing the model to confidently reinforce the wrong path.
- Chen recommends using AI to translate medical jargon, summarize records, generate appointment questions, and explore alternatives through follow-ups, while avoiding framing prompts around your suspected diagnosis.
- He warns that long-record summarization can fail through omissions, date/number confusion, and reinforcement of erroneous “chart lore,” making verification and careful context management necessary.
- Looking ahead, he expects more autonomous AI-driven clinical workflows (including limited prescribing/refills) and sees the biggest near-term impact as democratizing access to scarce medical expertise rather than “curing all diseases.”
IDEAS WORTH REMEMBERING
5 ideas“Human + AI” isn’t automatically better; integration and training determine gains.
Chen cites studies where GPT-4 alone beat physicians who had access to it, largely because many clinicians lacked chatbot fluency and because humans introduce anchoring and workflow friction. With training and better task design, human+AI improves—just not automatically.
Never lead the chatbot with your guess; feed it objective observations instead.
If you tell a model your suspected diagnosis (“Is this scabies?”), it tends to agree and elaborate, pulling you deeper into the wrong branch. He recommends describing only observable facts (symptoms, timing, photos, vitals) and asking for differential possibilities and what would change urgency.
AI’s biggest medical risk is omission and context confusion, not wild hallucinations.
Models often err by leaving out important details when summarizing long records (e.g., a lung nodule that needs follow-up). They also struggle with dates/numbers and can mix old issues with current ones (“context rot”), especially when you dump years of data at once.
Use AI to translate and prepare for appointments, not to self-prescribe.
A practical, high-value use is turning your labs/doctor notes into plain-language explanations, a concise timeline, and a question list for your next appointment—so scarce clinician time is spent on decisions rather than decoding jargon. This helps patients be better prepared without pretending the AI is the decision-maker.
Match AI use to the stakes: double-check when consequences are serious.
For low-stakes choices (e.g., minor OTC decisions), AI guidance may be “good enough,” but for high-stakes triage and major interventions (surgery, chemo selection), Chen urges verification with accountable professionals. He notes chatbots may also bias toward higher-acuity advice to reduce liability.
WORDS WORTH SAVING
5 quotesThe AI probably is smarter than most doctors, but knowing everything doesn't make you somebody who's worthy of trust.
— Dr. Jonathan Chen
If you don't have the medical context, the training, the subject matter, um, expertise, you might ask the question in the wrong way. You don't even know how to formulate it, and if you get down the wrong rabbit hole, the AI will very confidently, sycophantically agree with you and lead you all the way down the wrong path.
— Dr. Jonathan Chen
If you tell it what you're thinking, it's gonna have a very strong tendency to agree with you.
— Dr. Jonathan Chen
You have to say, "My daughter had a rash. She's scratching very heavily. Um, this has never happened before. This is what it looks like." And just, like, objective statements. Don't tell it what you're thinking.
— Dr. Jonathan Chen
It's you have competence, communication, and character, and you need all of them in one person.
— Dr. Jonathan Chen
High quality AI-generated summary created from speaker-labeled transcript.
