By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Peter Attia MD#143 - John Ioannidis, M.D., D.Sc.: Why most biomedical research is flawed, and how to improve it
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
At a glance
WHAT IT’S REALLY ABOUT
Ioannidis Explains Why Most Research Fails and Fixes
- Ioannidis explains his 2005 mathematical model showing that a nominally significant result (p just under 0.05) is more likely than not a false positive in most research settings, because of low prior odds, low power, bias, and many teams testing the same questions.
- He contrasts genetics, which reformed through large consortia, stringent thresholds, and accurate measurement, with nutritional epidemiology, which he says relies on biased questionnaires, post-hoc analyses, and entrenched beliefs.
- He and Attia discuss better ways to get answers on diet, including controlled trials, Mendelian randomization, and exposure-wide association studies, while noting that the PREDIMED trial's randomization problems show even top trials can be flawed.
- Ioannidis argues that questionable research practices are widespread but that the fix is training, better incentives, and unconflicted public funding for testing products rather than punishment or cutting research, with philanthropy and honest science communication also playing a role.
- He describes the Santa Clara and LA County seroprevalence studies, which found far more infections than documented cases, and the politicized attacks and harassment that followed, which he says endanger scientists' willingness to share findings.
IDEAS WORTH REMEMBERING
5 ideasA significant p-value often does not mean a true finding
Ioannidis says that once prior odds, power, bias, and multiple teams are accounted for, a result with p just under 0.05 has a greater than 50% chance of being a false positive in most circumstances.
Low power causes both false negatives and exaggerated effects
In under-powered settings, detected signals tend to be inflated and are more likely to be false, while big-data settings create the opposite problem of overpowered studies that pick up bias as significant results.
Genetics reformed itself in ways nutrition has not
Genetics adopted very large studies, whole-genome testing, thresholds like 10 to the minus eight, data sharing, and independent analysis teams, helped by accurate measurement and fewer entrenched beliefs.
More evidence is not automatically better evidence
Ioannidis warns that large meta-analyses assembled from teams invited because they already found an association can magnify cherry-picking and be more misleading than fragmented single studies.
Randomized trials in nutrition mostly fail to confirm observational signals
He notes over 200 large nutrition trials, with few exceptions, did not show the benefits suggested by observational data, which suggests most of those signals were false positives or exaggerated, though not all.
WORDS WORTH SAVING
5 quotesMagnitude and amount of evidence alone does not make things better. A- actually, it can make things worse. You need to ask what is the foundational construct of how that evidence has been generated and identified and synthesized.
— John Ioannidis
I think that if we continue incentivizing people to get significant results, no matter how that is defined, we are incentivizing people to do the wrong thing. We, we should incentivize them to, to try really interesting ideas and to have a high chance of failing.
— John Ioannidis
But, but I don't care about treating robots. I care about managing and helping real people.
— John Ioannidis
We need to defend our method. We need to defend our principles. We need to defend the honesty of science in trying to communicate it rather than build exaggerated promises or narratives that are not realistic. Then even if we do get the funds, we have just told people lies.
— John Ioannidis
We are just at the beginning of knowledge. And I feel like a little kid who just wants to learn a little bit more, a little bit more each time.
— John Ioannidis
High quality AI-generated summary created from speaker-labeled transcript.
