Read a Paper Like a Scientist: An ED Doctor’s Three-Question Framework for Health Information
The question that started this episode
A listener wrote in with one of the best questions I’ve received in months: “Dr Cois, do you have a framework that will help me read a medical paper?”
Yes. I do. And by the end of this post, you will too.
Here’s why this matters more than it ever has. For two years running, the World Economic Forum’s Global Risks Report has ranked misinformation and disinformation as the most severe short-term risk facing the world — above extreme weather, above armed conflict, above cyber attacks. The firehose of half-truths, cherry-picked studies, and confident influencers is not slowing down. And most of it is wrapped in just enough science-y vocabulary to feel legitimate.
Recent data on this is sobering. A Tebra analysis of viral medical content on TikTok found that videos with over five million views were 14 percent more likely to contain misleading advice than videos with under one million. Roughly one in four Gen Z respondents have turned to TikTok for medical advice instead of consulting a clinician. The platform with the worst signal-to-noise ratio is now a primary source of health information for an entire generation.
You do not need a PhD to push back on that. You need a checklist — the same checklist a first-year medical student is taught at journal club, and the same one I still use today on every paper I read.
The three-question framework
Question one. What kind of study is this, and where was it published?
Is it a perspective piece — one expert sharing an opinion? An animal study, where the “patients” are mice? A Petri-dish study, where the patients are a few cells on a slide? An observational cohort, where the authors followed real humans over years? Or a randomised controlled trial, where humans were randomly assigned to one treatment or another?
Each of these answers a different question, and each carries a different weight of evidence. A mouse study tells you what might happen biologically. A cohort tells you what does happen in a population. A randomised trial tells you what happens when you actually intervene. None of them is useless — but you have to know which one you’re looking at before you decide how much weight to give it.
Then check the journal. New England Journal of Medicine, JAMA, The Lancet, BMJ, and the Cochrane Database of Systematic Reviews are top-tier peer-reviewed journals. Smaller specialty journals can still be excellent. A preprint server such as medRxiv has had almost no peer review at all — that doesn’t make a preprint wrong, but it does mean you should read it more carefully.
Question two. Who paid for this, and who wrote it?
Every credible paper carries a funding statement and a conflict-of-interest disclosure. Read them. If a paper concludes that beef has no negative effect on cardiovascular risk, and the funding line says “supported by the Beef Checkoff,” that should give you pause. That doesn’t automatically make the paper wrong — but it raises the bar of evidence required to accept its conclusion, especially when it conflicts with the broader literature.
Then check the authors. Have they published in this field before? Are they at a credible institution? And — this is one of the most powerful tricks in evidence-based medicine — if the paper reports a clinical trial, head to ClinicalTrials.gov and pull up the pre-registered protocol. Trials are required to declare their primary outcome before they start. If the final published manuscript reports a different primary outcome and doesn’t explain why, that is a red flag for outcome-switching — a classic marker of investigator bias.
Question three. What is the PICO?
PICO is the workhorse of evidence-based medicine. Four letters. Memorise them and you will sound smarter than ninety percent of the comments section.
• P is for Population. Who was studied? Mice, humans, pregnant women, children, ICU patients, healthy 30-year-olds?
• I is for Intervention. What did they do? Which drug, which dose, which behaviour change?
• C is for Comparator. What was the control? A placebo? Standard of care? Nothing at all?
• O is for Outcomes. What did they measure? A blood test number? A heart attack? Death?
Same question, three papers, three different weights of evidence
Let’s pressure-test this on a real question that has been argued about for decades: does saturated fat raise your cardiovascular risk? There is a study at almost every level of the evidence pyramid that has tried to answer it, and they don’t all reach the same conclusion. Which is exactly the situation you face when you Google a health question.
Paper one — Ruuth and colleagues, 2021.
Population: a small group of healthy adults. Intervention: a few weeks of overfeeding with saturated fat. Comparator: overfeeding with unsaturated fat. Outcome: how “sticky” LDL particles became in the lab — a surrogate marker for cardiovascular risk. This is a mechanism study. It tells you saturated fat changes the behaviour of cholesterol particles in a worrying way. It does not tell you anyone had a heart attack. It points an arrow. It does not prove anything.¹
Paper two — the 2024 umbrella review of cohort and trial data.
Population: millions of adults across many countries. Intervention: lower saturated fat intake. Comparator: higher saturated fat intake. Outcome: actual cardiovascular events — heart attacks, strokes, deaths. This is a step up the evidence ladder. It pools the cohort data and shows that across enormous populations, lower saturated fat tracks with lower cardiovascular events. The signal is real, but cohort data can only show correlation, not causation.²
Paper three — the 2020 Cochrane review by Hooper and colleagues.
Population: adults across fifteen randomised controlled trials, totalling around 59,000 participants. Intervention: a dietary modification to reduce saturated fat. Comparator: usual diet. Outcome: cardiovascular events. This is the top of the pyramid. The conclusion: reducing saturated fat reduces the risk of cardiovascular events by approximately 17 percent. That is a robust, replicated, high-confidence finding.³
Now stack them up. Mechanism, epidemiology, randomised trial. The body of evidence converges. That convergence is what we mean when we say a finding is “evidence-based.” If you had only read paper one, you might shrug. If you had only read paper two, you’d feel pretty convinced. The Cochrane review gives you near-certainty. Same question. Three answers. The truth lives in the synthesis, not any single study.
Five red flags every reader should learn to spot
• “One study showed…” — One study is a starting point, not a conclusion. Real findings replicate.
• Surrogate outcomes dressed up as patient outcomes — A drug that lowers a blood test number is not the same as a drug that prevents a heart attack.
• Relative risk without absolute risk — “Cuts your risk in half” sounds enormous. Half of a 0.01 percent baseline risk is a rounding error. Always ask: half of what?
• Cherry-picking — If a creator cites three papers and ignores the other thirty that contradict them, that is not science. That is a debate-club tactic in a lab coat.
• Conclusions that don’t match the data — Read the results before you read the abstract. The abstract is the author’s pitch. The results are the actual story.
How Should This Modify Your Practice?
For patients and the general public
This week, find one health claim that has been circulating in your feed. Track it back to the paper. Run the three questions: What kind of study is it? Who paid for it? What is the PICO? Then ask yourself the most important question of all — does this paper actually answer the question I am interested in? You will be shocked how often the answer is no.
Bring the claim to your primary care doctor at your next visit. A good clinician welcomes the conversation — and the answer they give you, with the paper open in front of you both, will be far more valuable than the comments section.
For trainees and clinicians
If you are a medical student, resident, or practising clinician, this framework is your minimum viable journal-club preparation. Before you read a single sentence of the discussion section, write down the study design, the funding source, and the PICO. Compare the pre-registered protocol against the published outcomes. Look for outcome-switching, composite-outcome inflation, and per-protocol versus intention-to-treat reporting. These are the moves that separate a clinician who reads papers from a clinician who is told what papers say.
And teach this to your patients. Every counselling encounter where a patient brings in an Instagram screenshot is a teachable moment — and an opportunity to model the kind of disciplined thinking that will serve them long after the visit ends.
Author
Dr Adrian Cois, MD — Emergency Medicine physician, host of Overheard in the Emergency Room, and founder of DrCois.com. Special interests include preventive medicine, scientific literacy, and the application of evidence-based medicine to everyday patient decisions.
Disclosure: Dr Cois has no financial relationships relevant to the content of this episode.
Related Reading on DrCois.com
• Episode 14 — “Supplements, Peptides, and the Standard of Evidence” (Main Season)
• Overheard Journal Club Episode 1 — “SALT-ED: Balanced Crystalloids vs Saline”
• Episode 2 — “What the Dietary Guidelines Actually Say (and What They Get Wrong)”
References
1. Ruuth M, Lahelma M, Luukkonen PK, et al. Overfeeding Saturated Fat Increases LDL Aggregation Susceptibility While Overfeeding Unsaturated Fat Decreases Proteoglycan-Binding of Lipoproteins. Arterioscler Thromb Vasc Biol. 2021;41(11):2823-2836. doi:10.1161/ATVBAHA.120.315766
2. Effect of reducing saturated fat intake on cardiovascular disease in adults: an umbrella review. Frontiers in Public Health. 2024. doi:10.3389/fpubh.2024.1396576
3. Hooper L, Martin N, Jimoh OF, Kirk C, Foster E, Abdelhamid AS. Reduction in saturated fat intake for cardiovascular disease. Cochrane Database Syst Rev. 2020;5(5):CD011737. doi:10.1002/14651858.CD011737.pub2
