Stats, Context & Lampposts
While we venture no opinion on the illustrations that open this article, the broader point on statistical obfuscation is an important one and reminiscent of a quote attributed to Churchill: “Statistics are like a drunk with a lamppost: used more for support than illumination.” Statistics cannot be appropriately and carefully considered without knowing the context in which they were gathered and in which they are being reported/used.
When it comes to evaluating risk, there’s a big risk of choosing the wrong number.
Researchers typically measure the effects of, say, a risky health behavior or a controversial new law by comparing how two groups fare. There are two common ways to make the comparison, and the numbers produced can look very different — and be interpreted quite differently by the public. Each can be appropriate, depending on the situation and audience, but listing one without the other is a recipe for confusion, many statisticians say.
Two recent news stories demonstrate how two sets of numbers can describe the same situation. A study by public-health researchers, including several from Harvard University, found that eating one additional serving of processed red meat each day is associated with an increase in the risk of death each year by 20%. But that translated into only two more deaths per 1,000 people per year among those eating the extra meat, or a rise of 0.2 percentage points.
In a different sphere, the Department of Justice blocked a Texas law that would require voters to have identification, because Justice said Hispanic registered voters were at least 46.5% more likely to lack the necessary ID, which Justice cited as a discriminatory effect. Critics pointed out, though, that Justice was citing numbers that also show the proportion of Hispanic registered voters without the ID is just two percentage points higher — 6.3% against 4.3% — than the share of non-Hispanic voters without it.
That all these numbers describe the same situation reflects how different relative and absolute risk can look. Relative risk tells us how many times more prevalent an outcome or characteristic — death, or lacking an ID — is among one group compared with another. The absolute number tells us how much being a member of the group at greater risk increases that risk.
“Relative risk is the ideal measure for statistical analysis, for addressing scientific questions,” says Don Berry, a biostatistician at the University of Texas MD Anderson Cancer Center in Houston. “But it is irrelevant for individual decision-making.”