8 You say; I say; Who confirm?

Contributor: Shannon Ang
Date: 06 February 2020

People argue about how to interpret statistics. Often, parties quickly resort to inflammatory language and mudslinging in public, accusing others of being “disingenuous” and “dishonest”. A dichotomy is typically drawn - essentially, “my” interpretation is absolutely correct and you are “manipulating the numbers”. There are indeed cases where this can be a legitimate claim, such as erroneously interpreting a conditional probability as a total probability (see Chapter 4.3). But most cases are not so clear cut, and cannot be reduced to errors of technicality.

The interpretation of statistics is not context-free - this means that some ways of looking at numbers support certain arguments better than others. Both statistical rigour and domain expertise are needed to establish a strong link between evidence and conclusion. Simply drawing a dichotomy and taking an adversarial approach is very unhelpful when the disagreement is about what certain statistics tell us about larger phenomena (e.g., earth is getting warmer, people have more liberal attitudes).

In this case study, we will look at two very public cases of disagreements around statistics. I will suggest some ways in which we can move past the (unnecessarily) incendiary language being used, and ask the questions that really matter.