By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Peter Attia MD188 - [AMA #30 Sneak Peek] How to Read and Understand Scientific Studies
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
CHAPTERS
- 0:02 – 2:44
Why a Primer on Reading Studies Now
Peter Attia and Bob open by explaining why they are doing a Q&A on the structure of science. It grew out of a conversation with Tim Ferriss and the audience questions that have piled up over the past year. They frame it as a companion to the written Studying Studies series.
- •Peter recalls a podcast with Tim Ferriss, who pressed him on how to get better at making sense of studies
- •The earlier Studying Studies series was not very digestible
- •Bob gathered the past year's questions about understanding the structure of science
- •The Q&A format complements the written blog series rather than replacing it
- 2:44 – 3:29
Telling Solid Findings from Contradictory Headlines
Bob poses the core question of how a non-scientist can judge whether study findings are solid when studies seem to contradict each other, citing coffee and eggs. He sets up the run of questions, starting with how a study goes from idea to execution.
- •Contradictory headlines such as coffee good/bad and eggs good/bad motivate the discussion
- •The goal is helping people with no scientific background evaluate studies
- •The first question is how a study moves from idea to design and execution
- 3:29 – 6:39
From Hypothesis to Experimental Design
Peter walks through the steps before an experimental study is run. These start with a null and alternative hypothesis and continue through design, sample size, approvals, and funding.
- •Good science is generally hypothesis-driven, framed as a null hypothesis versus an alternative hypothesis
- •Experimental design covers duration, frequency, measurement, and ideally randomization and blinding
- •A power analysis determines how many subjects are needed
- •IRB approval, primary and secondary outcomes, a statistical plan, and pre-registration come before the study
- •Funding runs in parallel, and non-experimental studies skip some of these steps
- 6:39 – 7:57
Three Categories of Studies and the Evidence Pyramid
Bob asks about the different types of studies. Peter divides them into observational, experimental, and summaries or analyses of studies. He also explains his objection to the pyramid figure, which implies a meta-analysis always beats a randomized trial.
- •Studies fall into observational, experimental, and reviews or analyses of studies
- •The pyramid's hierarchy wrongly suggests meta-analyses are better than randomized trials
- •The figure's rainbow colors appeal to Peter's sons
- 7:57 – 13:43
Observational Studies: Case Reports, Case Series, and Cohorts
Peter explains observational designs using his own early case report on melanoma with high calcium. He then covers case series and retrospective versus prospective cohort studies, using saunas as an example. Bob notes that case reports can generate hypotheses.
- •A case report on a melanoma patient with high calcium showed how a single observation can be useful but not generalizable
- •Bob describes case reports as hypothesis generating, as with Steve Rosenberg's early motivation
- •A case series aggregates multiple similar cases, such as spontaneous cancer regressions
- •Cohort studies can be retrospective or prospective, such as sauna users versus non-users
- •Observational work can show patterns but struggles to establish causality
- 13:43 – 16:40
Experimental Studies: Randomized vs. Non-Randomized Trials
Peter distinguishes non-randomized (open-label) trials from randomized controlled trials. Non-randomized assignment introduces bias, and randomization is meant to remove it. Bob adds an example of patients choosing between a statin and ezetimibe.
- •Non-randomized trials assign people to groups for a reason, which introduces bias
- •Examples include antibiotics in the ER and surgery versus antibiotics for appendicitis
- •Randomized controlled trials are the gold standard because random assignment removes bias
- •Self-selection into treatments, such as statin versus ezetimibe, shows why randomization matters
- 16:40 – 20:38
Meta-Analyses and the Limits of the Pyramid
Peter describes how a meta-analysis weights and combines studies and how to recognize one by its forest-plot figure. He stresses garbage in, garbage out, and Bob adds that one rigorous trial could outweigh a meta-analysis of weak ones.
- •Meta-analyses weight studies by precision, so larger studies with smaller standard errors count more
- •A figure with rows of studies and hazard ratios with confidence intervals marks a meta-analysis
- •James Yang's line that a thousand sows' ears do not make a pearl necklace illustrates garbage in, garbage out
- •A meta-analysis should not be taken as gospel without examining each constituent study
- 20:38 – 25:24
Phases of Clinical Trials: Phase One and Two
Responding to Bob's question about phases, Peter explains the FDA process from preclinical work and the IND filing into phase one dose escalation for safety. He then covers phase two, which looks at efficacy, often open label, and Bob notes how non-randomization can introduce bias.
- •An IND application sets the intention to test a drug in humans after animal work
- •Phase one is a small dose-escalation study to assess toxicity, not efficacy
- •Phase two continues to assess safety and starts to look for efficacy, often open label with comparison to natural history
- •Bob notes patients who want a known treatment may self-select into non-randomized phase two trials
- 25:24 – 28:32
Phases Three and Four: Pivotal Trials and Post-Marketing
Peter explains phase three as a large, randomized, blinded trial that serves as the gold standard for efficacy and underpins approval. Phase four takes place after approval and can expand indications or reveal missed side effects, as with semaglutide for obesity.
- •Phase three involves potentially thousands of patients, is randomized and blinded where possible, and is longer
- •The larger sample can reveal side effects not seen earlier
- •Approval rests mainly on phase one, two, and three data
- •Phase four is post-marketing and can look at new indications such as semaglutide for obesity
- •Different populations may carry different risk profiles
- 28:32 – 31:42
Judging Confidence in Observational Studies
Bob asks what increases or decreases confidence in observational studies. The sneak peek ends here, followed by a membership pitch for the full episode.
- •Bob asks what to look for in observational studies to tell a pearl necklace from garbage
- •The preview cuts off before an answer is given
