Trusting statistics
I had a Facebook discussion yesterday about statistics. At one point, my interlocutor posted “You should know better than anyone that statistics can be manipulated to actually show the opposite of what is real.” Well, just to be clear, that’s not a problem with statistics, that’s a problem of motivated reasoning, which is not a statistical algorithm.
The topic of our discussion was economics, and he says he trusts Krugman on economics. Krugman, of course, uses statistics to draw his conclusion. He’s a Nobel Laureate, but the Nobel Prize in Economics is notoriously political. Here’s a little lesson is substituting argument from authority for argument from evidence.
Ronald A. Fisher is widely considered to be the founder of modern statistics. There’s no question that our understanding of the world owes a significant debt to Fisher and his statistical innovations. But Fisher was quite vocal in questioning the association of smoking with cancer, which is based on statistical association studies. Why did his genius fail him on the matter of epidemiological data for smoking, when there’s no record of him opposing any other epidemiological association? In all likelihood, motivated reasoning. Fisher was a smoker, and he was unable to deal with the cognitive dissonance posed by his habit vs the clear risk implied by statistical analysis of smokers. Ironically, Fisher died after colon cancer surgery. Risk of colon cancer in smokers is roughly the same as having a first degree relative with colon cancer.
There are two morals to this story: (1) the problem with statistics isn’t statistics, it’s the misuse of statistics, and (2) even the smartest authorities can be victims of motivated reasoning. As St. Ronnie was fond of saying, trust, but verify.
When genius errs: R.A. Fisher and the lung cancer controversy, PubMed (nih.gov)
Modern science, especially physics, relies on statistics to predict events, even when precise causes can’t be worked out. Also true of election predicting. This may not seem entirely satisfying; it is what it is.
Harry Frankfurt agrees with you:
“on Truth”
https://www.penguinrandomhouse.com/books/55573/on-truth-by-harry-g-frankfurt/
There is a formula to manually transform the link into the title
Joel
I am sincerely glad that you understand this. It is a little more difficult to apply it to one’s own “motivated reasoning.”
I do think people who say “you can’t trust statistics” mean “you can’t trust the statistics that people use to convince you.”
from my experience, statistics itself is hard enough, that one should be careful of claiming it proves anything.
‘Economics’ as a set of problem is at least as complex as ‘The Traveling Salesman Problem, which has no known solution currently. Too complicated.
Wikipedia: Traveling Salesman Problem
Supercomputers aren’t much help, supposedly quantum computers (which are essentially statistical) supposedly will work it out. Will any person be able to understand such solutions?
@Fred,
You may well be right. Statistics has far wider application than economics. There are plenty of folks who use statistics in my field–biomedical research–who misapply statistical tools and/or draw erroneous conclusions.
Physicist John Bell (with his eponymous ‘Inequality’) ‘proved’ that Quantum Entanglement was a real thing, and did so statistically, which is considered quite a breakthrough.
No one really understands how it comes about. But that’s statistics for you.
Wikipedia-Bell’s Theorem