Information Literacy Friday: Statistics is Hard

“Undeniable math”, “Statistical impossibilities”: these words do not mean what the non-mathematician-authored articles you may have seen them in think they mean.

One of the bigger problems I see in reporting, even with otherwise-excellent reporters in generally-reliable sources, is bad applications of or misunderstanding of math, particularly statistics. Add some bias on the part of the author—particularly where anything involving people or politics is involved—or a tight deadline, and it only gets worse. 

So, the information literacy lesson is: if you see someone making claims with math, and you, personally, can’t do all the necessary math to verify those claims, you should find someone trustworthy who can, or take them with a healthy dose of skepticism. There are several risks here:

1. they might not understand the math, themselves
2. they might be able to understand it, but didn’t fact-check in this case
3. they might /think/ they understand it
4. they might explain it badly (even if they understand it)
5. they might understand the raw math, but be misapplying it to this situation

…and there are probably other problems that can show up. And note that none of these problems assumes the writer is trying to deceive you. But badly-understood or badly-presented math can certainly play a part when someone does want to mislead you. 

This doesn’t mean that you can never accept math as proof if you don’t have a Masters in statistical analysis. It just means you need to be cautious, double-check and cross-check claims like these, and seek out people who are experts—which generally the article authors aren’t, unless you’re reading a math or science journal.