Erroneous Scientific Data - It’s Worse Than You Think

Remember during Covid when everyone had a study to support their particular viewpoint? I am guilty of that as much as the next person.

I just came across someone who illuminated to me why so many studies can contradict other studies. Aaron Brown, a quantitative expert and author of "Wrong Number: How to Extract Truth from a Blizzard of Quantitative Misinformation," has spent decades uncovering how flawed statistics become accepted as fact across industries. His research is particularly relevant to healthcare, where bad numbers directly impact patient care, policy decisions, and resource allocation.

Brown's core insight: expert institutions confidently present misleading data all the time, and neither journalists nor other experts catch the errors. In healthcare, this problem is especially acute—and especially dangerous.

Brown's book opens with the story of a 1975 gasoline rationing project where two separate experts confidently stated opposite facts about tractors.

One said 75% run on diesel, the other said 75% run on gasoline.

Neither had checked. Both were influential advisors. One from the US Department of Agriculture. The other from Ford Motor Company who made most of the tractors back then. Both refused to reconcile their contradictory findings.

The same pattern emerges in healthcare. A landmark Lancet study claimed U.S. AID saved 91 million lives between 2002 and 2020—a number that exceeds the entire global decline in mortality during that period by 14%.

No major media outlet questioned it.

The prestigious journal accepted it. Healthcare policy shifted based on data that, by Brown's standard, is mathematically impossible.

Similarly, a marijuana-heart disease study linked to peer-reviewed papers claiming telephone survey data were "accurate and reliable”.

The trouble was when you click on the links for the papers, they said that telephone survey data was NOT AT ALL accurate. Peer reviewers never clicked the links to verify. The study was published anyway.

Brown identifies a crucial dynamic: experts want to teach, not learn. They have their models, their numbers, and they're sticking to them. In healthcare, this becomes dangerous because the stakes are human lives.

Aaron Brown emphasizes that when someone claims a policy "saves X lives," they're almost always presenting only one side of the ledger. Healthcare organizations constantly announce that new protocols will save lives—but they rarely ask what happens to the $9 billion that could have been left in taxpayers' hands, or what alternative treatments might have done with those resources.

Brown's principle: always ask "compared to what?"

He also discovered that peer reviewers for major journals don't actually verify citations, replicate calculations, or check basic facts. They ask whether a paper is "important" and "original"—matters of opinion. A healthcare paper proposing a new diagnostic protocol might be published without anyone verifying the underlying methodology against existing literature.

His book also documents how studies present data from convenient periods that support a conclusion, then truncate the analysis before contradictory trends appear. A healthcare initiative shows dramatic improvement over months 1-6, but by month 12, the improvement disappears. You only hear about months 1-6.

Once prestigious institutions (medical journals, regulatory bodies, major hospital systems) endorse a flawed finding, challenging it becomes professionally risky. Nobody wants to be the person who says the emperor has no clothes.

He recommends skepticism toward data that exhibits these patterns:

1. If a healthcare breakthrough is just released to the media without a broader context, especially if it contradicts the normal way of doing things - be very skeptical.

2. When healthcare numbers seem impossible - like 91 million lives saved, they usually are.

3. A protocol claims to "save lives" without addressing what else could those resources accomplish? This violates Brown's cardinal rule: "always compare to the alternative."

4. Click the links. Check the references. Brown found that peer reviewers don't do this. Neither do journalists. Do your own research!

5. If improvement data stops conveniently after a short time window or before a contradictory trend appears, the analysis is incomplete—and suspicious.

6. When two authorities state opposite conclusions with equal confidence, neither may be trustworthy until someone investigates which is actually correct.

Makes you wonder who you can trust anymore?