I need to give a presentation to a group of students on Tuesday about why one needs to be careful when examining statistics or mathematical results in the media or online.

In his book How Not To Be Wrong, Jordan Ellenberg provides a few examples that I planned on using as case studies to present to the students

  1. Wisconsin governor in 2011 claims, since there was a net 18,000 jobs added in 2011 and 9,000 were in Wisconsin, that implies that Wisconsin is doing something right. Failed to mention that net jobs added included states that lost jobs, which reduces the net rate.
  2. Mathematicians find proof that the Torah sends messages to the future by looking at sequences of characters that correspond to rabbi names and said rabbi birth/death dates. However, these results only held in the event of very specific names and dates; using any other accepted names or dates for each rabbi resulted in failed tests.

If anyone knew of any other good real world examples of misleading statistics or mathematics. I know of a many examples due to variability in sample size, but the more intricate and (potentially) nefarious, the better.

Thank you!

  • 1
    $\begingroup$ A real example of Simpson's paradox: en.wikipedia.org/wiki/…. $\endgroup$ – Minus One-Twelfth Apr 12 at 23:52
  • 1
    $\begingroup$ The low birth weight paradox surprises people, as do any instances of Simpson's paradox $\endgroup$ – lulu Apr 12 at 23:52
  • 1
    $\begingroup$ See the book How to lie with statistics. (caveat: Have not read this myself.) $\endgroup$ – Jair Taylor Apr 13 at 0:26
  • $\begingroup$ There are many examples on Andrew Gelman's blog. You could start by searching there for "mine noisy data" and "forking paths" . You can point your audience to the blog where they can browse for themselves. statmodeling.stat.columbia.edu/2019/04/12/… $\endgroup$ – Ethan Bolker Apr 13 at 2:53
  • $\begingroup$ You could mention margins of error: Data like GDP are based on samples, and hence are estimates, and the margins of error are themselves only probabilistic. Mass media almost never mention this. And even if a margin of error is known for certain to be at most $\pm 0.06 %$, measurements of (e.g) unemployment rates of 6.00% and 5.9% cannot be known to imply a real difference. $\endgroup$ – DanielWainfleet Apr 13 at 4:31

In 2018 WWF published the Living Planet Report. This report was widely misinterpreted, and many newspapers reported something along the lines of a 60% reduction in wildlife since 1970 without giving a more detailed (or correct) explanation.

WWF ellaborated how their numbers were obtained in a technical supplement to their report. They write:

"Does the trend in the global LPI mean we have lost 60% of all animals? Although the LPI uses time-series of either population size, density, abundance or a proxy of abundance, the overall trend calculated represents an average trend in population change and not an average of total numbers of individual animals or species lost."

They even give an illustrated example, found below, which I have edited slightly to make more compact. Example WWf

  • $\begingroup$ A certain economist named Lomberg, who has been personna non grata to environmental scientists for many years, also manipulates data. E.g. When discussing wildlife he argues that the population of large animals in the USA has increased in the past 100 years. He's right, if you include chickens, pigs, cows, cats, and dogs. $\endgroup$ – DanielWainfleet Apr 13 at 4:41
  • $\begingroup$ So from what I understand, the misconception was that we lost 60% of everything, when in reality we lost a lot of a few species, while some lost relatively little. Is this correct? $\endgroup$ – wjmccann Apr 15 at 6:48
  • $\begingroup$ @wjmccann many of the chickens aren't as much wildlife as they are property of a factory that's feeding them until they can be made into nuggets. $\endgroup$ – JJJ Apr 22 at 0:24

Another example is (bitcoin) by trading volume. Sometimes people will say X (some number) of bitcoins are traded each day, that's the equivalent of Y (number depending on X and the exchange rate) million US dollars!

Sometimes, this is meant to impress, look at how many millions of dollars are traded each day, it must be important.

In reality, even if the figures are correct, it's a baseless number because it doesn't take into account who are parties to the transaction. Even if a lot of money is moved between different accounts, doesn't mean it's actually changing between owners. This is particularly the case with bitcoins because accounts are cheap (only the transaction costs money) so people can easily have large numbers of accounts.

A similar situation would be if you went to the ATM, withdrew 10,000$ and went into the office to deposit it back into your account. Then, to go back to the ATM, etc. etc. until you have withdrawn a 100 times that day. Only to brag to your friends that you are very important because you handled one million dollars that day.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.