Skip to content
Silicon Valley GirlSilicon Valley Girl

Stanford Doctor: Never Tell AI What You Think Is Wrong With You

Bring your team’s research, context, and AI workflow together on one canvas: Try Miro – https://miro.pxf.io/2RLQv0 Dr. Jonathan Chen is a Stanford physician and computer scientist who teaches doctors how to use AI. His team found that GPT-4 alone outperformed doctors using it on diagnostic reasoning tasks. But being smarter doesn’t automatically make AI trustworthy. In this episode, we break down how to ask AI about your health, use your medical records to prepare for appointments, and avoid steering a chatbot toward the answer you want to hear. Jonathan also explains why AI can leave out critical information, why more health screening isn’t always better, and what doctors still need to do when a computer knows more than they do. Timestamps: 00:00 — Is AI smarter than your doctor? 01:38 — How AI outperformed doctors using AI 03:33 — How your questions can steer AI toward the wrong answer 05:12 — Using your medical records to prepare for appointments 08:27 — What AI can help with and when to double-check 11:28 — How to ask about symptoms without leading the AI 12:57 — Which AI should you use for health questions? 15:45 — Does telling AI “I’m a doctor” help? 20:48 — Can you give AI too much medical history? 24:05 — Are full-body scans and extra tests worth it? 28:21 — When genetic testing can make a difference 29:28 — Can AI help detect cancer earlier? 33:31 — What to ask AI when doctors can’t find an answer 35:15 — Can relying on AI make you lose your skills? 36:51 — What makes a good doctor when AI knows more? 39:39 — AI doctors and automated prescriptions 41:39 — Will AI cure all diseases in 10 years? 42:24 — What better access to care could change 44:52 — Who should make the final decision? 47:10 — Always use multiple AI models for a second opinion Links: 📩 Follow my Newsletter:https://siliconvalleygirl.beehiiv.com/subscribe?utm_source=youtube&utm_medium=video&utm_campaign=futureproof-sub&utm_content=JonathanChen 🔗 My Instagram: https://www.instagram.com/siliconvalleygirl/ 📌 My Companies & Products: https://partnerships.marinamogilko.co

Dr. Jonathan ChenguestMarina Mogilkohost
Sep 25, 202648mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:00 – 1:38

    Is AI smarter than your doctor?

    1. JC

      The AI probably is smarter than most doctors, but knowing everything doesn't make you somebody who's worthy of trust.

    2. MM

      This is Dr. Jonathan Chen, a Stanford physician, computer scientist, and director of AI and Medical Education at Stanford Medicine. His study found that ChatGPT alone made better diagnoses than doctors using it.

    3. JC

      Maybe the human is getting in the way. Am I just slowing the computer down?

    4. MM

      A lot of people will start trusting AI because it's more sympathetic.

    5. JC

      Unlimited patience. Someone suing ChatGPT for, because it gave bad medical advice. It told me everything was fine, I stayed home, and I could have died because of it.

    6. MM

      But AI is so good at detecting small things, no?

    7. JC

      Yes, which means it can also get confused by noise.

    8. MM

      Curing all diseases. A lot of people talk about that in the next 10 years.

    9. JC

      Oh, you're gonna make me say spicy things.

    10. MM

      You just co-authored a randomized trial with 70 practicing doctors, and doctors who saw the AI before forming their own opinion performed significantly better. Why does this happen?

    11. JC

      It's a really bizarre phenomena. The technology's [laughs] moved very fast in the few, past few years, as you probably noticed. You know, six years ago I had students studying this thing, like, the, the language models, they were just really not very good. They could barely pass even the basics. And then a few years ago, we're like, "Holy crap. The capabilities are advancing very rapidly." And now for complex medical knowledge, a thing we used to pride ourselves, our identity as a physician and why we think you come to us as patients, is because we, we have a lot of the knowledge, and boy, is that just not the answer anymore. And it's really causing some existential angst about what is our relevant role and how do we provide useful value for patients. And a lot of people, including a lot of tech venture people who I work with, think, "Aha, you don't need doctors anymore 'cause the AI is smarter." And it's like, well, well, it is very powerful, but there's a little bit more than that in terms of what you probably

  2. 1:38 – 3:33

    How AI outperformed doctors using AI

    1. JC

      need for good healthcare.

    2. MM

      But still, uh, in another study, AI on its own outperformed doctors, and doctors who used AI couldn't catch up. So the question is, do you need doctors? [laughs]

    3. JC

      Uh, uh, the, the short answer is yes, but you might be different what do you need them for. So that was a, a lot of s- news from our article a couple years ago in New York Times, AI Defeats Doctors at Diagnosis. We were not trying to show that. That was not the purpose of our study. Our purpose was, oh, GPT-4 at the time, yeah, it must be a great thing. Let's give doctors access. I bet they'll be even better at medical diagnosis. And then it made them a little bit better, but the really weird thing, which we did not set out to find, it just came out of the study, was wait a minute, GPT-4 by itself was better than the doctors who have access to the GPT-4. This is a very weird, uncomfortable finding, 'cause it really flies in the face of what we call the fundamental theorem of informatics, right? Human plus computer, when you combine the two, will deliver better results than either alone.

    4. MM

      Which is not true.

    5. JC

      Does that sound so good?

    6. MM

      Right, in your study.

    7. JC

      It's just so appealing.

    8. MM

      Mm-hmm.

    9. JC

      It just intuitively feels right, and we're, like, empirically showing, wait, that didn't actually happen. And it leads to some very uncomfortable, um, feelings of maybe the human is getting in the way. Am I just slowing the computer down? Maybe I should just roll over and let the robots take over. That's actually not what I think it means. There's a lot more to unpack. Some of it is really, like, you know, a calculator's just better at calculation. Why would you compete with a calculator at long division? That just doesn't make sense. Some of it was also clearly an education issue. A third of the docs in the study had never used a chatbot before in their life. They had no idea how to use it. Another third maybe used it once or twice. And so in follow-up ones where we, like, kinda taught the people how to use it in real time, now the combo was better. Still not better than AI by itself, but also it's what kind of tasks and questions. If it's question answering, if it's knowledge retrieval, even something that looks like reasoning, AI's getting pretty freaking good. It's getting pretty freaking good, and that's why health is the most popular type of question for AI chatbots. Um, and yet you still need more than that for complete healthcare, just like you need more than that for what is an educator, for example.

  3. 3:33 – 5:12

    How your questions can steer AI toward the wrong answer

    1. MM

      So why is this happening? Is that the way we ask questions? 'Cause-

    2. JC

      Mm-hmm

    3. MM

      ... what I understood is that-

    4. JC

      Mm-hmm

    5. MM

      ... when we're asking questions, we're kind of hinting-

    6. JC

      Yes

    7. MM

      ... the answer that we wanna hear.

    8. JC

      Mm-hmm.

    9. MM

      And that's why AI sometimes works worse-

    10. JC

      Mm-hmm

    11. MM

      ... with us.

    12. JC

      It, it definitely is, right? So we know there's a little bit of prompt engineering, but don't overdo it because it'll get incorporated in the base models. There's a great study, it's not my study, came out, um, Nature Medicine a bit ago, where it was like our study where doctors answering medical questions, but instead forget doctors and nurses and it, they had regular people without medical training answer a medical triage question. Here's the scenario: Should you go to the emergency room or not, for example. And at the time it was the GPT-4s and Llama 30, you know, by the way, we're way past that, right? We're generations beyond that. Does very well. Regular people don't do well. I mean, why would they? They have no medical training. How could they know how to answer a medical question? But aha, aha, let's democratize expertise. We'll give the people AI and they'll be better off. And in that study they found that the regular people who had access to AI, they did worse. They did worse than if you didn't give them AI. They would've been better off not using it at all. What, what's going on here? It's because if you don't have the medical context, the training, the subject matter, um, expertise, you might ask the question in the wrong way. You don't even know how to formulate it, and if you get down the wrong rabbit hole, the AI will very confidently, sycophantically agree with you and lead you all the way down the wrong path. It really has that amplifying capability. I describe this about machine learning, it's like a mirror to ourselves. If you're doing something good, doing good medical diagnosis, it'll make us better about that, but if you're doing something that you didn't intend to that wasn't so good, it will just amplify your ability to do the wrong thing, too. And so clearly, very powerful alien intelligent technology landed on the planet, and just dropping it in people's lap doesn't actually solve all our problems. We have to think about how to incorporate

  4. 5:12 – 8:27

    Using your medical records to prepare for appointments

    1. JC

      it.

    2. MM

      So let's talk about how ordinary people should be using AI for-

    3. JC

      Sure

    4. MM

      ... healthcare questions. Let me, [laughs] I don't know, brag a little [laughs] -

    5. JC

      Go for it. Go for it

    6. MM

      ... how I worked on my AI set-up this summer, 'cause I was doing all the checkups.

    7. JC

      Mm-hmm.

    8. MM

      And I have my lab since, I think, 2019, and I have this high-

    9. JC

      Great

    10. MM

      ... cholesterol problem.

    11. JC

      Yeah. Mm-hmm.

    12. MM

      So, like, I, 90% [laughs] of us-

    13. JC

      Yeah. I'm, I'm starting to get high blood pressure, too, and now I'm, like, trying to eat healthier.

    14. MM

      Oh, right? But i- in my case it doesn't even work. Like, d- whatever.

    15. JC

      Mm.

    16. MM

      So I was seeing a lot of different doctors 'cause I wanted to hear opinions.

    17. JC

      Yep.

    18. MM

      And I created this ... for Perplexity chat with all my labs-

    19. JC

      Good

    20. MM

      ... for the past-

    21. JC

      Okay

    22. MM

      ... seven years, uh, with dynamics-

    23. JC

      Mm-hmm

    24. MM

      ... and everything. And then, uh, I inserted whatever doctors ever told me, and whenever I came to... Oh, and my 23andMe raw data-

    25. JC

      Okay

    26. MM

      ... uh, combined-

    27. JC

      Okay

    28. MM

      ... with the labs. And so Perplexity, by the way, it was like right away it found like a genetic thing. I was like, "Wow, this is so cool." So any doctor that I saw, I just came and like I told Perplexity, "I'm about to see this cardiologist."

    29. JC

      Mm-hmm.

    30. MM

      "What should I be asking them-

  5. 8:27 – 11:28

    What AI can help with and when to double-check

    1. JC

      Yeah.

    2. MM

      Exactly.

    3. JC

      Yeah.

    4. MM

      So what would you say, how do we use AI in the right way so that we get-

    5. JC

      Yeah

    6. MM

      ... the right information and not, uh, end up in the, this pastor's situation?

    7. JC

      I think as long as you're aware of the caveats, a lot of powerful potential... And it's, it's this very weird duality of people talking out of both sides of their mouth, right? On the one hand, there's disclaimer on these things. "This is not medical advice. Talk to your doctor." On the other hand, "Look at this cancer patient whose life we saved." Like, you, you kind of, kind of have it both ways. You gotta realize what's actually going on in our regulatory landscape. Like, our human medieval institutions can't keep up with the pace of technology movement. So I think it's actually a great tool to explain, "Here's my, I, here's my medical record. Here's my doctor's note. I have no idea what all this jargon is. Can you explain this to me, what all this means and what are the implications?" Actually really pretty good for that. General medical knowledge, these things are actually really, really quite good already. If you get to like specialty level knowledge, eh, you know, you're picking which chemotherapy to pick, you know, I, I think you better get some more opinions, including AI actually in that mix to kind of organize your thoughts around that. Um, it's this weird dynamic of like therapy, counseling, and advice. It is obvious already it's happening more. People are gonna receive that from automated bots than live human beings. And probably it's not 'cause the human beings are so bad. It's just, it's just not available, right?

    8. MM

      Yeah.

    9. JC

      We'll never keep up with the-

    10. MM

      Yeah

    11. JC

      ... unlimited demand for such services, so you're gonna reach for what you can get your hands on. When the stakes matter, uh, double-check wh- when you need to. And sometimes that's actually what doctors are used to dealing with 'cause we deal with like trainees, consultants all the time, very smart people. And, you know, if it's Tylenol versus ibuprofen, you know, it's, it's probably fine.

    12. MM

      Yeah.

    13. JC

      But if you're picking which surgery to do, you know, [laughs] it actually really matters we get this one right. Let's double-check that one.

    14. MM

      We are extending our team, and there is one thing we're trying to improve right now, onboarding new producers for this podcast. Giving someone the files is really easy. Giving them the context behind the decision is so much harder. Why did we choose this interview angle? Why did another one underperform? What does a strong question list look like for us? A lot of that knowledge usually lives in the heads of people who've been doing this for years. So for one of our recent episodes, we tried an experiment with Miro, who is sponsoring this video. So instead of sending a new producer a folder of docs and then opening Claude separately, we connected Claude through Miro's MCP server and put everything onto one canvas. Previous research, examples, audience comments, and things we've learned from past interviews. So instead of expanding the backstory again, we could ask Claude, "Why did we choose that episode angle?" or, "What are we still missing?" Then we used flows to turn that brainstorm into the next stage of work, strongest angles, research gaps, possible questions, and next steps. That's what I like about this setup. Claude does the reasoning while Miro gives it the context the team has already built and gives everyone a place to work on the output together. If your AI workflow is starting to turn into disconnected chats and docs, check out Miro through the link in the description. But even with small things, I had this issue like a month ago. My daughter woke up scratching, and I-

    15. JC

      Oh,

  6. 11:28 – 12:57

    How to ask about symptoms without leading the AI

    1. JC

      okay

    2. MM

      ... I took a photo, asked ChatGPT, and ChatGPT was like, "Scabies. Go to the emergency room."

    3. JC

      [laughs] Okay. Yikes.

    4. MM

      I'm like, "Whoa, okay."

    5. JC

      Mm.

    6. MM

      And I called our doctor-

    7. JC

      Mm-hmm

    8. MM

      ... and I sent her the picture. She's like, "No, allergy."

    9. JC

      Yeah.

    10. MM

      That's like-

    11. JC

      Yeah, yeah

    12. MM

      ... that's not scabies.

    13. JC

      Yeah.

    14. MM

      And I'm like, "Okay, this was interesting-"

    15. JC

      Mm-hmm

    16. MM

      ... 'cause I think ChatGPT overreacted.

    17. JC

      Mm-hmm.

    18. MM

      So can you tell me, what is the right way to ask those questions to not get overreaction and to not get False negative

    19. JC

      Oh, gosh. That's, um... It's, it's, you know, judgment versus knowledge, right? Um, some of it is you know, about sycophancy, your audience does too. If you tell it what you're thinking, it's gonna have a very strong tendency to agree with you. We've shown this in our studies in both directions. If AI is so smart, should it write the first draft and you edit it, or do you write the first draft and let it, like, double-check for you? Both can work, but there's sycophancy and anchoring bias in both directions. So you have to say, "My daughter had a rash. She's scratching very heavily. Um, this has never happened before. This is what it looks like." And just, like, objective statements. Don't tell it what you're thinking.

    20. MM

      So no, no guessing.

    21. JC

      Yeah.

    22. MM

      Yeah.

    23. JC

      Don't say-

    24. MM

      Yeah

    25. JC

      ... "Is this some really nasty infection that I should go to the [laughs] emergency room for?"

    26. MM

      [laughs]

    27. JC

      Then it's just like, "I don't know. Maybe the emergency room [laughs] sounds interesting," right?

    28. MM

      Yeah.

    29. JC

      It's gonna want to follow your train of thought.

    30. MM

      Yeah.

  7. 12:57 – 15:45

    Which AI should you use for health questions?

    1. MM

      Yeah

    2. JC

      ... for example.

    3. MM

      Would you say there are some chatbots that are better at that? 'Cause somebody told me Grok is so much better at [laughs] health questions. I didn't really, like, notice a huge difference.

    4. JC

      Mm-hmm.

    5. MM

      Uh, but, I mean, it, it, it would give you more information-

    6. JC

      Mm-hmm, mm-hmm

    7. MM

      ... if you're asking. Have you noticed that?

    8. JC

      You, you may have noticed, it's a weird thing, right? 'Cause they're all just computers. We anthropomorphize them. We treat them like they're humans. They have different personalities, right?

    9. MM

      Mm.

    10. JC

      Um, they're... All the foundation of the frontier model's actually pretty good at general medical knowledge 'cause they've digested the internet, right? And there's a lot written about general medical information, specialty knowledge so much. I have noticed some are more, [clicks tongue] I don't know, what, what's the word? Uh, prudish about talking about some topics.

    11. MM

      Mm.

    12. JC

      The irony being people won't talk to it then, and it'll... They'll go to somewhere else. And, and so I can imagine Grok might be more forthwith about just saying things that others are like, "You know, you should probably talk to your doctor about that. There's too many context situations." Uh, the kind of concrete, uh, side example, I remember early on, I was trying to ask ChatGPT, "Make 10 versions of a joke about neurosurgeons," 'cause I'm gonna use it in my talk, and then it came up with 10 jokes, nine out of 10 were really bad, but one, one has potential here. And I tried Gemini. Let's see how Gemini does. And, "Make 10 jokes about neurosurgeons," and it refused. [clicks tongue] It ref-

    13. MM

      Oh.

    14. JC

      That's not very nice to make jokes about neurosurgeons. They're very hardworking, good people. Like, "Dude, I don't [laughs] have time."

    15. MM

      So-

    16. JC

      "So I'm going back [laughs] to the other one-"

    17. MM

      Yeah

    18. JC

      ... "if that's how it's gonna be." [laughs]

    19. MM

      Which AI would you recommend for our health-related questions?

    20. JC

      So there are medical specific ones. Uh, some of them are designed for doctor use specifically, um, whether it's Glass Health, Amboss, Doximity, Open Evidence. There's, there's many of these, but for regular people, often you don't have access to those. You need to have a, a medical license number. So really, any of the frontier models, I, I still use ChatGPT all the time.

    21. MM

      Mm-hmm.

    22. JC

      Claude is actually pretty good.

    23. MM

      Have you tried Health?

    24. JC

      Gemini's pretty good.

    25. MM

      'Cause it's, it's brand new. I think they released it a month ago. I haven't used it either 'cause-

    26. JC

      Yeah

    27. MM

      ... I just don't understand the difference between-

    28. JC

      Yeah

    29. MM

      ... like just uploading all your labs-

    30. JC

      Yeah

  8. 15:45 – 20:48

    Does telling AI “I’m a doctor” help?

    1. JC

      tools as well.

    2. MM

      Okay. Somebody told me, somebody who mentioned that Grok is the best-

    3. JC

      Mm-hmm

    4. MM

      ... that the best way to prompt AI when asking a health-related question-

    5. JC

      Mm

    6. MM

      ... [laughs] is to say, "Hey, I'm a doctor. I have a patient right here. [laughs] Here are their symptoms."

    7. JC

      Mm-hmm. Mm-hmm.

    8. MM

      "Can you tell me what are the... Like, what would you suggest doing with this patient?" Is that a good way to ask? [laughs]

    9. JC

      It, it depends on your context. That is a great way. You know, is it like, oh, do I prompt engineer to death to make it more accurate? That doesn't help. Uh, you say, "Pretend you're a super genius," versus to, "Pretend you're incompetent." It actually still tends to give the same type of information, but again, it'll change personality and style. So if you say, "I'm, I'm a doctor," or you pretend to be a doctor and explain it, it'll formalize the way it answers, and that can be very helpful in getting structured responses rather than a kind of flattened, warm response.

    10. MM

      Would it give you more information if you ask that way?

    11. JC

      It, it could, right. It'll give you more detail, and if you're an analytical person, like I bet many of the people listening to this, this podcast are, right? They actually want that very structured, detailed, analytical information. Most regular people out there, which I learned a lot through my medical training, oh, like, there are different people who live different kinds of lives. They don't want highly technical, organized, structured information. It's actually overwhelming.

    12. MM

      Mm.

    13. JC

      They want it to be more accessible, um, in a more general way. And so it... You tell it the format you want it to be, and then it is remarkably good at that. Then you're not asking it to think necessarily or make a decision, 'cause that's higher stakes, but just, just translate this. Just explain this in a different format that I would better appreciate, uh, these tools are quite good at.

    14. MM

      Mm-hmm. What if, uh, something like, um, the story that I told you about scabies, what if something like that happens at 2:00 AM? How would you know that AI is overreacting?

    15. JC

      That's, uh, that's a tough one. That is what I tell my medical trainees, is like, knowledge, you're not gonna beat AI at knowledge. What you need to have is judgment. You have judgment, and two things can be true at the same time. You, your daughter is itching, and it might be scabies. These two things are true at the same time, and it might and probably isn't, but it's at 2:00 AM, and you don't have a doctor. All of those things are true at the same time. Um, in that moment, what is so powerful, I actually think the revolutionary capability of these chatbots is not that they know everything, they can help your efficiency. All that is great. It's, um, that you can have this dialogue You can follow up. It's like, "Well, if it's not scabies, what else could it be? How concerned should I be about this? Are there key things I should be looking for?" The fact that you can ask follow-up questions is so much powerful than looking up an article online.

    16. MM

      Yeah.

    17. JC

      It's like, "I read the article, but it doesn't tell me about my situation."

    18. MM

      Yeah.

    19. JC

      Um, that's actually a very powerful situation. You might say, "Well-

    20. MM

      So assume the opposite.

    21. JC

      Yeah.

    22. MM

      I like that approach. Mm-hmm.

    23. JC

      A- and that's a recurrent one. Just tell me, "Is- so I th- it's scabies. I should go to the ER, right?" Then-

    24. MM

      Yeah

    25. JC

      ... the chatbot can say, "Maybe you should."

    26. MM

      Yeah.

    27. JC

      Or if you say, "Well, convince me why it's not-

    28. MM

      Mm

    29. JC

      ... and then let me think about that," then that's a very powerful way to get it to be this, uh, sparring partner I've heard it described as, is a better way to help you through tense situations where the correct answer is not obvious, let alone ethical dilemma situations where there is no correct answer. It really is a dilemma.

    30. MM

      Yeah.

  9. 20:48 – 24:05

    Can you give AI too much medical history?

    1. JC

      likely things.

    2. MM

      Yeah, and this, this could be the problem-

    3. JC

      Mm

    4. MM

      ... with something that I mentioned as my setup. When you upload so much information, then it can omit something that's important when I see a doctor.

    5. JC

      Oh, yeah. Well, if you wanna go there, we're finding context rot is a real issue, and, um, dates and number perception. So Stanford, we're one of the places first that has ChatEHR. In so many words, we took something like ChatGPT and we plugged it into our Epic electronic medical record. So the AI can just read your, the entire elect- medical record for you and synthesize that. Very powerful, very cool. It got released a lot faster than I think maybe it [laughs] should have, but people were so desperate. "Please help, we're drowning in paperwork." Um, in reviewing it, it is very powerful, but a few common mistakes it can make. It doesn't know numbers. It's a language model, right? It's autocomplete on steroids. It doesn't know what a number is. It really doesn't know what a date is. So you could give it 100 notes, throw all of your medical records, all your labs, all your genetic material for the past 10 years, it will very easily get confused and start talking about something from seven years ago which just doesn't apply or isn't relevant today anymore. But it's all just this one pile, and it's very easy for you to confuse. O- you overwhelm the context or you get context rot. One other interesting error that I found in the way it behaves is, um, chart lore. I don't know if you've heard of what this is. Once a diagnosis, and sometimes a stigma-associated diagnosis, is attached to your medical chart, somebody says it once, almost every doctor has a very strong tendency to just copy and paste it forward.

    6. MM

      Mm.

    7. JC

      And you will never get rid of that diagnosis on your chart. It's extremely hard to do so. Um, so I had an example of a guy. He was, he was a homeless man, and we were taking care of him for some other reason. And people said he's probably schizophrenic, because there's this definitely association between being homeless and being schizophrenic. Except when psychiatry actually evaluated him, it's like, "No, he's an odd guy who's homeless. That doesn't mean he's schizophrenic. He's never been on medication." But every doctor just kept copy and pasting homeless schizophrenic man. And then the AI, if you ask summarize, it also says homeless schizophrenic man, because that's what everybody else is saying. Even though in theory it has access to that psychiatrist note. It could have dug back and looked it up, confirmed otherwise, but it is confidently parroting back what everybody else is saying too.

    8. MM

      One thing before we keep going. If you wanna stay updated on all the things these doctors, researchers, and founders say here, we take the use cases and prompts from every conversation and put them into my newsletter called Future Proof. And of course, we also put my own beautiful use cases there. For example, we put together the seven skills that came up again and again, the ones people will need in the next few years. These are the skills that my team and I are also using daily. Subscribe through the link in the description. You'll get that one, and you can also read our previous newsletters and see what you missed. Okay, knowing all, all this, can we wrap up and draw a perfect picture for someone who wants to use AI for their health? What should their chat or whatever it is look like?

    9. JC

      Oh, gosh. Um, I, I think a continuous coach, an ongoing dialogue, um, helping you explain, like, things about your health and where to go. Um-I think that's a great use. I really do.

    10. MM

      So upload all your labs, right?

    11. JC

      Get, get your actual doctor's real note, right? So then there's, like, ob- more objective information. Say, "Can you explain this to me, and what should it mean for my life, and what are reasonable things?" The reality, it's pretty good. I can't officially say that that is good medical advice, because there's always some caveat, but in practice, I know people are gonna use it anyways, so we better acknowledge, meet people where they

  10. 24:05 – 28:21

    Are full-body scans and extra tests worth it?

    1. JC

      are.

    2. MM

      Would you say there are some labs that everybody should do, or, like, a 23andMe test-

    3. JC

      Mm-hmm

    4. MM

      ... or a full-body MRI scan-

    5. JC

      Mm

    6. MM

      ... to get, like, an established-

    7. JC

      Ooh. Oh, you're gonna make me say spicy things.

    8. MM

      [laughs]

    9. JC

      [laughs] Okay.

    10. MM

      It's just we're in Silicon Valley.

    11. JC

      Yes, I know. [laughs]

    12. MM

      People are obsessed with their health.

    13. JC

      I know.

    14. MM

      And you hear all of these things, and sometimes you think, "Oh," and sometimes you hear a story.

    15. JC

      Mm.

    16. MM

      Somebody done a full bo- body MRI scan, and then-

    17. JC

      Mm-hmm

    18. MM

      ... a year later they did another one and they saw something growing-

    19. JC

      Mm-hmm

    20. MM

      ... and this is how they prevented it.

    21. JC

      Oh, wow. That, that's... Okay, you're dragging me to a deep area, but it, it's a good one to talk about, especially for this audience. On the one hand, that's compelling, right? The one MRI found that somebody, saved their life 'cause they cut out the cancer. That really is great and powerful. They're not talking about the other thousands of people getting random scans and blood tests, checking for early cancer and really not finding anything. Almo- most all of these early disease detection things, if you're asymptomatic, if you're otherwise healthy, um, almost always these are gonna fail and be a dead end. Almost always. There's very few cases. You know, people... We have breast cancer screening, we have colon cancer screening because they've been studied in a very broad way. We've wanted to screen for pancreatic cancer. By the way, I have, like, five people who have died of pancreatic cancer in my family. I wish we could screen for it, and it just, it just doesn't work. It doesn't work. None of the tests have been accurate enough or reliable enough for you to do something about it.

    22. MM

      Even, like, all the new Galleri, the Galleri test? No?

    23. JC

      You... They're not accurate enough and they don't detect it early enough for you to do something different about it. Um-

    24. MM

      Wow

    25. JC

      ... ovarian cancer, another one. We've wanted to screen for it for decades, and just we've tried it. It just, it, the, the numbers don't work out when you look for rare disease. I'll say the big caveat here. I'm, I'm not dismissing all of these diagnostic approaches. If it's screening healthy people who are otherwise fine, the word well in Silicon Valley, perhaps some people watching this podcast, um, B, the healthy things are actually quite obvious and it's just they're not the fun things to hear about. It's like healthy diet and exercise. Um, if you're asymptomatic, mostly enjoy your life. If you're symptomatic, that's very different.

    26. MM

      Mm.

    27. JC

      I have a family history. My mom and my dad both died of a heart attack in their 40s. Like, okay, you should probably go get... Uh, they did.

    28. MM

      Yeah.

    29. JC

      I'm just saying that hypothetically.

    30. MM

      Mm.

  11. 28:21 – 29:28

    When genetic testing can make a difference

    1. JC

      secret sauce.

    2. MM

      So you said 50% lifestyle, 30% genetics. So would you say uploading your raw data from your genetic test will help your doctor?

    3. JC

      Uh, this is [laughs] a very loaded thing. I have a lot of colleagues who work very deep in, like, pharmacogenomics, this kind of thing. It is interesting... Oh, gosh, I'm trying to be very careful to be helpful without offending people.

    4. MM

      Yeah.

    5. JC

      Uh, I think it's very interesting. For some people it can be very high-leveraged genetic diagnosis. If you find out you're at risk for, um, familial hypercholesterolemia. So y- it's not like regular people who have high cholesterol. It's like, no, you have a genetic issue. You, you're not overweight, but your cholesterol's through the roof. You're gonna have a heart attack before you're 50, almost certainly, unless... And yet we have medications that really can fix that situation if only it was detected, and most people won't detect it until their first heart attack, right? You, you don't have a symptom because you have high cholesterol. You don't feel it. You don't have an ache, you have a pain. It's just one day you have a heart attack, right? And that's, that's by the time you notice.

    6. MM

      Mm.

    7. JC

      So if your genetic testing can spot things like this, it can make a huge, huge difference in your health. Um, the majority, 90% of the population, it'll be interesting, but probably not enough to steer you in a very high-leverage way.

    8. MM

      Not actionable.

    9. JC

      Yeah.

    10. MM

      Okay.

  12. 29:28 – 33:31

    Can AI help detect cancer earlier?

    1. MM

      And when you said there are some cancers that we still can't screen for, like pancreatic cancer-

    2. JC

      Yeah

    3. MM

      ... do you think AI has a shot there to help us in foreseeable future, in our lifetime?

    4. JC

      It definitely is gonna help. It's definitely gonna help. Um, some of it is, uh, a small numbers problem, but maybe there's subtle signals, and that is the, the hope for a lot of these things. If you measure one blood test, if you do one ultrasound, one MRI, you know, it's actually kind of not that much information. But what if you tracked all your genetic biomarkers and protein, uh, signals over the course of 10 years you're alive? You can track the trends, and then we have masses of data we can, uh, synthesize that. Which, humans, there's no way you can do that. You need AI, you need machine learning, you need other, um, data analytic techniques. Then you might find, wait a minute, there is a signal here, and we can detect who's at risk, you know, 10 years advance. If you detect pancreatic cancer one year advance, like, it's, it's still too late. You're, you're not gonna do anything different about it. Uh, if you do 10 years in advance, maybe you can do something about it now that wouldn't have been feasible. Um- Uh, again, I'm trying to be optimistic, it's just that this is such a seductive idea about the early detection and, um, it just almost never works out the way you think it would.

    5. MM

      Mm.

    6. JC

      Because if you could detect it 10 years early, it's probably too small to matter, and, uh, what would you do?

    7. MM

      But AI is so good at detecting small things, no?

    8. JC

      Yes, which means it can also get confused by noise, and oh, you're ma- you're making me get into some very, [laughs] very loaded areas, but-

    9. MM

      [laughs]

    10. JC

      ... but it's, it's good. Um, bre- breast cancer screening is great, right? Breast cancer's a terrible disease, and a lot of our screening methods have earlier detection, finding all these small lesions, and yet survival numbers of actual people who are, are survivors of breast cancer haven't actually changed that much because ironically, a lot of the really small lesions we caught, your immune system would've just killed it off anyway. You didn't have to do anything. If you s- find, find a small prostate cancer, very common, the reality is if you do nothing, most people will be fine.

    11. MM

      Mm.

    12. JC

      Their body can take care of it. Your body's actually... Your biology's a lot smarter than any AI we've ever created or ever will. Um, and also our treatments have improved, so even if we catch a disease a little bit later, we're often very good about being able to treat and intercept. So if you put this all together, I think we should try to find better early detection, a whole health span treatment, precision health our dean likes to call it, to kinda go after it, just realizing it's, it's a lot more work than it would seem to really make it work and deliver the outcomes that we want for our patients.

    13. MM

      So would you say the biggest impact of AI that you're seeing right now in healthcare is this ability to analyze all the information and s- maybe spot patterns?

    14. JC

      So the obvious way to me, it's democratization of access to the scarcest medical resource, which is expertise, somebody who knows what they're doing. Like, you can manufacture drugs, you can manufacture hospital beds. It doesn't, doesn't matter if you don't have people who know what to do with it. So that i- to me is my naive techno-optimist perspective. Now, I'm very biased, right? Because that's basically what I research, is, um, access to expertise, someone who can think through your case, reason through it, answer your questions, know which medication to pick, know when you don't need a medication. That's very powerful difference. When I saw early version of GPT-4, what, three years ago, I was like, "Holy crap, this thing just leapfrogged so many technologies in the field much faster than, than I've expected." I threw away half my research program. There's just no point in trying to invent some technology here 'cause it's basically already arriving. Let's jump ahead to how people can use it, 'cause that also is part of the change. You know, alien technology doesn't do anything by itself. Combine that with humans who know how to use it, now it's very powerful. Uh, a different track of what I actually think the revolutionary capability of, um, AI chatbots at least, we haven't even talked about agents yet, is that you can simulate these conversations. Pretend to be the doctor, I'm the patient. Change it around, you pretend to be the doctor, so you pretend to be the patient. Pretend to be my mom. Pretend to be my coworker. Pretend to be my business negotiation partner and do practice these conversations. Um, that I think is wow, very eye-opening for empathy and processing and getting, reasoning through complex, difficult situations, that you were never able to do something like that before with technology. You know, does it help you analyze records? Does it help you do your writing, increase your efficiency? That's fine. It's good. It does all that, but you've never had something like that where you can practice this dialogue in a way that's

  13. 33:31 – 35:15

    What to ask AI when doctors can’t find an answer

    1. JC

      quite powerful.

    2. MM

      Can you recall any, uh, cases in your practice when doctors could not establish the, a diagnosis, but then they used AI and they found the reason? Um, were there any cases like that? And then how can ordinary people learn from that? If they're seeing multiple doctors, nobody can really point out what's the problem, what can they do?

    3. JC

      You, you got it. I think there really is an opportunity here, and then, you know, undiagnosed diseases or delayed diagnosis is actually a very common issue. And you know, your doctor with all best intentions, we're not jerks, we're not dismissive, although I know it can come across that way. It's like, we got 15 minutes here, you know? What am I supposed to do? I gotta do my best. And then if I play the odds, if I say it's just high cholesterol, high bloo- if I say it's just anxiety or something that's very common, I'm probably right, right? I'm probably right. I, I pick the most likely things. Uh, or do I think you have rare genetic syndrome that causes, uh, obscure pancreatitis? I'm probably [laughs] wrong if I guess that too often. Um, so once you've seen multiple doctors in a row, they're making their best effort, but they're not giving you a really satisfactory answer, I think that is a good idea. Say, "Hey, AI, I don't expect you to tell me anything surprising, but just here's what the info is so far. Is there anything anybody might be missing?"

    4. MM

      Mm.

    5. JC

      "Are there other considerations I should look to? Is there another test that might be useful to ask my doctor about?" Um-

    6. MM

      That's a good prompt right here.

    7. JC

      Yes.

    8. MM

      Yeah.

    9. JC

      Yeah. That, that, that, that really is. That really is.

    10. MM

      And you're not suggesting anything.

    11. JC

      Mm-hmm.

    12. MM

      You're not leaning towards any doctor.

    13. JC

      Mm-hmm.

    14. MM

      You're just asking-

    15. JC

      Yeah

    16. MM

      ... for another opinion.

    17. JC

      Another great way to do is before you go into that doctor's appointment, it's like, "Hey, here's my info. What questions should I ask my doctor? What are the four bullet points that are useful for them to know?" So that y- you don't waste your time or their time just, like, rehashing the info.

    18. MM

      Mm-hmm.

    19. JC

      It's like, "Oh, that's a great summary. Now we can focus on the decision-making and the context, the processing," that I think will really

  14. 35:15 – 36:51

    Can relying on AI make you lose your skills?

    1. JC

      help.

    2. MM

      From what we discussed, uh, in the medical field-

    3. JC

      Mm-hmm

    4. MM

      ... how can we apply this, and do you think it's applicable to a general knowledge worker?

    5. JC

      Mm-hmm.

    6. MM

      Because the way you prompt, the way you ask questions, it already tells AI what you wanna hear. And there is actually, um, another thing that can be borrowed from the medical field. Experienced endoscopists actually performed worse-

    7. JC

      Mm-hmm, mm-hmm

    8. MM

      ... without AI after getting used to it.

    9. JC

      Yes. This is a very compelling study, and it's not the only one. It's, uh, many examples, like this is not my study. I just thought it was a fun thing. So endoscopy, it's a, a stomach doctor. They do a colonoscopy. They look up your bottom with a camera, and they're looking for polyps, these little things that, that could turn into colon cancer. What we do, we spot it and cut it out years before it turns into cancer, and this is where you really can save people lives. This is a screening tool that actually, uh, saves lives. Um- But it's hard to see this thing, right? You're looking at this camera, these little spots. Aha, aha, AI, computer vision. Real time, it scans the image and it kind of puts a little box, "I think this is a polyp." And the doctor says, "Oh, wow, that really helps me. They're finding even more polyps. They're even better at detecting these subtle lesions." And then later in the study they turned off the com- the computer vision system and the doctor was still practicing. They're actually worse. They're worse than before they had the technology available. They got used to it. They started to depend on the technology, and they became worse off capability-wise than before they ever started. So de-skilling, never skilling, are all very real issues that we have to come to grips with. What are the skills we are okay with delegating to a computer, which means we probably will be less good at it, and which are ones we better retain as foundational? A- and this really is an existential question that, um, I don't think we have a simple answer to at this

  15. 36:51 – 39:39

    What makes a good doctor when AI knows more?

    1. JC

      point.

    2. MM

      Do you personally have an answer to it?

    3. JC

      [laughs] Uh, I've been working to articulate that one. In the past year I finally articulate what makes a doctor a doctor, a clinician, a professional a professional in any specialty. It's you have competence, communication, and character, and you need all of them in one person. Any one of those is great at all, uh, eh- you need all three to really be a professional. There are some skills, you know, like do you compete with like a calculator at long division? No, you just give up. You say, you know... Do you remember your best friend's phone number? I forgot that a long time ago, um, 'cause you just, you trust your computer to do that, your smartphone to do that, versus is it reasoning, judgment, communication? The reality is a computer can do a lot of that. It really can. Uh, there is a too easy mode. I've had so many doctors say, "Yeah, yeah, yeah, but hey, I have the human touch, okay? I mean, I, I can build relationships with my patients." I'm like, "I know you can, and that's good. It really does make you better," but people build relationships with AI, AI chatbots right now, and they actually will talk to them about more things that are more comfortable. As a director for medical education in AI at Stanford, and we need a policy. What should our students, what do they need to learn and what should... You know, don't, don't, why, why make them memorize facts? That's not the right skill for them to learn. But knowing how to use judgment and apply them and communicate those, I think those are still foundational.

    4. MM

      I think judgment forms based on your knowledge.

    5. JC

      Yeah.

    6. MM

      If you don't have enough knowledge, how do you develop judgment?

    7. JC

      I love it. I love it. So yes, um, many people think, or these students go, "Why do I have to study this? It's stupid." And I get it. Like, there's no way I'm beating an AI in memorizing these 100 bugs, 100 drugs. It's obvious computers should be better at that. And yet still, and yet still students say you still should study and have knowledge.

    8. MM

      Yeah.

    9. JC

      Because how are you gonna get to the higher order judgment if you don't even know the vocabulary of what we're talking about?

    10. MM

      And it looks like it's becoming even more important, because remember like a couple years ago there were people saying, "Oh, education is gonna be obsolete. We don't need to learn all the facts."

    11. JC

      Mm-hmm. Okay, good. Mm-hmm.

    12. MM

      And now when we understand that judgment is becoming a primary skill, then it looks like everyone should [laughs] have this master's and master's to develop judgment, or at least go very deep in their niche to be able to judge, not just possess knowledge.

    13. JC

      I, I mean, I'm aware I'm in the ivory tower here, so I'm biased, but I love that response. I love that. Yes, it would be great for all. I, I also realize that maybe that doesn't apply to everybody's life, but I think there's a lot of power and utility there. It, you know, Bill Gates had this quote a year ago. He goes, "AI is gonna, within 10 years, will replace many doctors and teachers. Humans will not be needed for most things." And I'm like, I'm a little disappointed in him 'cause he really should know better about how, um, economics works. But I would say, you know, if, what is a teacher? If a teacher is someone who transmits the information and who has the answers to your questions and grades tests, then if that's all a teacher is, actually yeah, a computer probably could do that. But being a good educator is more than just give you the information. You can mentor people, you can advise people, you can role model for them. That is actually, I think, a very different thing than a source of

  16. 39:39 – 41:39

    AI doctors and automated prescriptions

    1. JC

      knowledge.

    2. MM

      If we're looking, say, three to five years ahead, what is the breakthrough in AI and medicine that will happen that people are not talking about enough?

    3. JC

      Oh. It, well, it, it's, it's a weird thing, right? 'Cause I'm in the Silicon Valley ivory tower bubble, right? And so I hear about it all the time, and I realize when I go out and give talks to hundreds of doctors and people out in the community, it's like, oh, wait a minute, this isn't common knowledge. Uh, AI doctor type things, a computer that can just prescribe medicines. Three years ago I would never have said this out loud.

    4. MM

      Oh, already happened.

    5. JC

      Yeah.

    6. MM

      I got my, I got my-

    7. JC

      Yeah

    8. MM

      ... whatever it was. I had, uh-

    9. JC

      Mm-hmm

    10. MM

      ... dermatitis on my forehead.

    11. JC

      Mm-hmm. Mm-hmm.

    12. MM

      And like I-

    13. JC

      Yeah

    14. MM

      ... clicked a couple buttons and I got my prescription.

    15. JC

      Mm. Yeah.

    16. MM

      Which is like still to me not that convincing, just because in any other country I just go to a pharmacy.

    17. JC

      [laughs] Yes, yes. There you go.

    18. MM

      I don't know why I have to go to a chat to like-

    19. JC

      Yes

    20. MM

      ... ask and beg for this. [laughs]

    21. JC

      Yeah. But if this is a topical thing, a lot of mundane thing. But a higher order stuff is, is, it's inevitable, it's coming. And there's many ARPA-H programs we're involved in. Uh, the FDA, the Human Health Services are all engaged in the conversation. How do we get to autonomous AI medical practitioners? What is the landscape we need to do to get to that? I would not said this out loud three years ago 'cause it would've made people angry. It would've-

    22. MM

      Mm

    23. JC

      ... it sounded scary. And there literally are like protests and strikes among like Kaiser nurses and this kind of thing. You know, trust nurses, not AI. There's this very vibes of just-

    24. MM

      Mm

    25. JC

      ... anxiety and uncomfortableness. Even though it's like, um, now I'd say it out loud, not only because it's obvious, inevitable, because like it's already happening. It's already happening. There's a regulatory experiment in Utah where Doctronic, Legion Health, other companies, they'll just do medication refills, and you never interact with a human, and no human double-checks it either. It is purely an automated computer system while it refills. But it's just refills.

    26. MM

      Refills is easier. Yeah.

    27. JC

      It's just only certain medications that are relatively safe, but that's, you know that's just the Trojan horse, right? [laughs] That's just the foot in the door.

    28. MM

      Yeah.

    29. JC

      Once they've got there-

    30. MM

      But it's interesting, like people-

  17. 41:39 – 42:24

    Will AI cure all diseases in 10 years?

    1. JC

      more than that-

    2. MM

      Curing all diseases. A lot of people talk about that in the next 10 years.

    3. JC

      I, I'm, I'm... Oh, you're gonna make me be such a downer, aren't I? [laughs]

    4. MM

      [laughs]

    5. JC

      I wanna go, a great vision and aspiration, uh, b- but, but no. Um, you know, right? Biology is still outsm- smarter than, than all of, uh, uh, the AI in the world and, and will be. So a lot of new advances, new drugs, new discoveries, new ways to screen, new ways to cure, I think those are very plausible. Not necessarily new, but like you will get to them faster because we can do-

    6. MM

      Mm-hmm

    7. JC

      ... the research more expediently and manage the much more complexity. I think that's very optimistic. I think you've got a front door to your doctor's office much more expediently. That's a thing that you could imagine and I bet it's gonna be kind of ... boringly normal in just a few years, uh, because, uh, it's just so easy to deliver it now in ways that before was kind of impractical.

  18. 42:24 – 44:52

    What better access to care could change

    1. MM

      What are you personally looking forward to most?

    2. JC

      Um, I, I do like the democratization of access. I like that a lot. As, you know, I'm, I'm, I'm a PhD in computer science. I used to be a software engineer. I'm a nerd, because I like to work on computers for fun. But I went into medicine, healthcare, 'cause I wanted applications that really affect human health i- in that way. And I am always frustra- I can't do the job the way I want to do it. I only have 15 minutes to talk. I know it would help if I talked, you know, the half-hour. I just can't. I have 20 other patients to see, right? That's very frustrating to me. I, I found, I found your diagnosis. Aha, I found it. You have this hormone problem. I'll send you to the hormone doctor. Six months for an appointment. That's so frustrating-

    3. MM

      Yeah

    4. JC

      ... 'cause I actually know what they need, and I can't even get the patient what they need. Um, so this ability to get people what they need in an expedient way, and all sorts of people who never even made it to the doctor's office were not really taken proper care of, um, I think that's very great to look forward to.

    5. MM

      What has been the best breakthrough for you?

    6. JC

      There's too many. Well, most of my team is now on, um, a- agentic workflows. I finally, I was just, uh, I keep trying to keep up. I started playing with, like, Claude Cowork and these type of things a few months ago. And I don't know if you know about me, but, like, I'm literally a multi-award-winning magician. I like to perform magic tricks, right?

    7. MM

      Mm-hmm.

    8. JC

      It's an illusion.

    9. MM

      Mm-hmm.

    10. JC

      I started playing with these, like, holy crap, this is actual magic. [laughs]

    11. MM

      [laughs]

    12. JC

      This is actual magic.

    13. MM

      Yeah.

    14. JC

      It's not a chat to dialogue. It's like, I don't want to answer these emails anymore. I don't want to fill out these forms anymore. I don't want to-

    15. MM

      And it just does it

    16. JC

      ... update my calendar. And now I have an AI agent which g- has its own separate email account for, like, privacy security management. Uh, but as I did them more, it was like a brand-new toy. Like, holy crap, what else can this thing do? And I'm still trying to expand upon there. You'll just have to try it. I can explain. It sounds, like, okay. But once you get your hands on it, it's like, "Whoa, crap." And then you, you won't, you will not be able to go back.

    17. MM

      Do you have your own personal health dashboard as a doctor?

    18. JC

      Uh, y- you know, ironically, the classic thing, as a doctor, I'm one of the worst patients of all. [laughs]

    19. MM

      [laughs]

    20. JC

      It, it... what finally motivated me, I've been pretty healthy most of my life, but, um, I got, uh, a colonoscopy for health screenings a few, a couple years ago. And then I was just there waiting for my procedure, and then the nurse just basically checked, "D- d- you have blood pressure 155/90. You have high blood pressure." I'm like, "Oh, dang it." [laughs]

    21. MM

      Ooh.

    22. JC

      It's, it's like, "You have to go to your primary care doctor. You gotta get all your testing. You gotta get on medicine." And I'm like, "I really don't wanna go." So this, this is irony. As a doctor, I'm actually a terrible patient. And I said, "Okay, fine, I'll, I'm gonna diet and exercise right now. I'm gonna try to manage it on my own." But I do have a spreadsheet now that keeps track and tries to keep me accountable, which different people respond different ways. Uh, most people, a spreadsheet will not work. People need a, a different kind of system.

  19. 44:52 – 47:10

    Who should make the final decision?

    1. MM

      Do I understand this correctly, is that AI really helps you summarize information, come up with additional questions you should be asking-

    2. JC

      Yes

    3. MM

      ... and things you should be paying attention to-

    4. JC

      Mm-hmm

    5. MM

      ... because you would never guess. But then the final decision-making, critical thinking, and deciding-

    6. JC

      Mm

    7. MM

      ... what to do is still you and your doctor, or maybe multiple doctors you're seeing. 'Cause like me, I [laughs] I'm always looking for a doctor with a more agreement [laughs] more-

    8. JC

      Yes. [laughs]

    9. MM

      ... like with AI. [laughs]

    10. JC

      Interesting, right? You're gonna doctor shop, right? AI shop and doctor shop.

    11. MM

      [laughs] Yeah.

    12. JC

      Um, I think those are very, very good principles. Um, and I've gotten pushback on that commentary where it's like, "Why do you still need to double-check with a doctor?" I'm like, "Well, it depends on the stakes of what are involved." 'Cause it's not 'cause the doctor is smarter, by the way, that the AI's pretty freaking smart. It really is. But it has better, full awareness of the context and how it affects your values, your preferences, your risk tolerance, right? Um, are you willing to, like, "You know what? It might shorten my life by one year, but if I eat that bacon, it makes me happier, [laughs] so I'm just gonna eat it," you know, for example. Um, these are things that, um, can still contextualize things in a different way. Great principles. Just be aware of their caveats. I actually think I'm overly, I'm naively optimistic here. If you're aware of the caveats, you can do all sorts of great things. Just be careful. There are people who, strangely, who, who, w- who the people who are vulnerable are the ones who don't know how to use that full judgment to catch the mistakes, and end up accidentally hurting themselves like that former pastor. And there's many other stories which we've seen in the news of people who have, like, self-harm, like really-

    13. MM

      Yeah

    14. JC

      ... dark things where-

    15. MM

      Especially with mental health.

    16. JC

      Yes.

    17. MM

      'Cause it just, again, because it wants to please you, and if you mention something, it's gonna follow that path.

    18. JC

      Exactly.

    19. MM

      And God forbid if you ask the, um-

    20. JC

      Mm-hmm

    21. MM

      ... like, a mental health question in your business thread-

    22. JC

      Mm-hmm, mm-hmm

    23. MM

      ... or as, like, something that's unrelated, it's gonna be even worse.

    24. JC

      Yes.

    25. MM

      I stopped actually doing that, 'cause every time I got an answer, I was like, "No."

    26. JC

      Yeah, yeah.

    27. MM

      "No, that's weird."

    28. JC

      It's you getting concerned. A lot of vulnerable people are not gonna be able to exercise that key judgment in a way because they're having, uh, some other issue. So I'm trying not to be flippant about it.

    29. MM

      Mm-hmm.

    30. JC

      I think it's so powerful to be aware of the caveats. You really can do some good for yourself and others. But it's like a chainsaw, like a chainsaw.

  20. 47:10 – 48:33

    Always use multiple AI models for a second opinion

    1. MM

      great.

    2. JC

      One thing, we have this no harm study right now, which is, like, checking medical consultations. Does AI give u- forget useful, correct. Is it at least, is it even harmful or safe? Um, and there's still har- harmful advice that can come out of AI basically that we found across all of them. But another interesting thing is, um, [clicks tongue] um, multi-agent, which is basically a second opinion.

    3. MM

      Mm-hmm.

    4. JC

      Your, your one doctor tells you to start a cholesterol medicine. It's like, is that really right? You ask another doctor because you wanna double-check, and there's a reason for that. Because every human, they're, we're really smart. We're better than random, but it's still easy to make mistakes still.

    5. MM

      Yeah.

    6. JC

      So the more opinions you have, you smooth out this, uh, rough- roughness. Strangely, that works with AI, too. ChatGPT says something, go say, "Gemini, can you double-check that? Claude, can you double-check that?" And now it asks all of them, "Can you synthesize that?" Um-

    7. MM

      In Perplexity, you can do model orchestration.

    8. JC

      In Per-

    9. MM

      Yeah.

    10. JC

      Perfect.

    11. MM

      Yeah.

    12. JC

      Okay. And it, it'll do that for you. And strangely, maybe it's not that strange, maybe it's intuitive, um, you will get better, safer answers if you do that.

    13. MM

      100%.

    14. JC

      Yeah.

    15. MM

      Jonathan, thank you so much. You're very knowledgeable. Thank you-

    16. JC

      Yeah

    17. MM

      ... for sharing this knowledge, and, uh, thank you for being positive about AI.

    18. JC

      Uh, absolutely. There is, uh, I, I actually think overall very, not positive, predictable harms. We manage them, we can really reap a lot of the benefits.

    19. MM

      Yeah. Thank you. If you want to stay ahead in the AI era, subscribe to this channel. New episode every week on AI careers and how not to get left behind. Thank you for your support.

Episode duration: 48:33

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode oOpF0ZXSXuk

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

iOSAndroidClaudeChrome