What a study actually proves

Correlation isn't causation

Many health headlines come from observational studies, which watch what people already do and look for patterns. They're useful for spotting signals, but they can't show that one thing causes another.¹

One example from the NIH: a study found that older Mexican American adults who drank chamomile tea had a 29% lower risk of death than those who didn't. That doesn't mean the tea saved lives. Tea drinkers may differ in many other ways, like being more active, and statistics can only adjust for the differences researchers measured.¹

A randomized controlled trial, which randomly assigns people to a treatment or a comparison, is the strongest way to test whether something actually works.

Relative and absolute risk

Headlines often report relative risk: "doubles the risk" or "cuts the risk by 30%." That's the change compared with another group. The absolute risk is the actual chance. Both matter, and the two are often misunderstood.²

For example, if a risk goes from 2 in 1,000 people to 4 in 1,000, that's a 100% relative increase, "doubles the risk," but an absolute increase of just 2 in 1,000. When you see a percentage, ask: out of how many?

Questions to ask about a study

• Was it in people, or in animals or cells?
• Was it a randomized trial, or observational?
• How many people were in it, and who were they?
• Is it one study, or a review of many?
• Who paid for it?
• What do professional guidelines say?

Who was studied

For decades, women were left out of much of the research medicine relies on. In 1977 the US FDA advised caution about including women who could become pregnant in early drug trials, and the guidance ended up keeping women of reproductive age out broadly.³ In 1993, the NIH Revitalization Act made including women in NIH-funded research mandatory.³

The gap still shows. Women have side effects from drugs nearly twice as often as men, and most drugs in use were approved based on trials done mostly in men. In one review, 76 of 86 drugs reached higher levels in women's blood, and that predicted their side effects.⁴ From 2013 to 2023, just 8.8% of NIH grant spending went to women's health research.⁵

Myth: If a study found it, it's proven.

It's natural to trust "a new study says." But a single study is one piece of evidence. It may be small, in animals, observational, or not repeated. Advice changes as evidence builds, and that's science working, not failing.