Proving causation: causality vs correlation

Does going to the hospital lead to an improvement in health? At first glance, one might assume that visiting a hospital should improve health outcomes. However, if we compare the average health status of those who go to the hospital with those who do not, we might find that hospital visitors tend to have worse health overall. This apparent contradiction arises due to confounding – people typically visit hospitals due to existing health issues. Simply comparing these two groups does not tell us whether hospitals improve health or if the underlying health conditions of patients drive the observed differences.

A similar challenge arises when examining the relationship between police presence and crime rates. Suppose we compare two cities—one with a large police force and another with a smaller police force. If the city with more police also has higher crime rates, does this mean that police cause crime? Clearly not. Instead, it is more likely that higher crime rates lead to an increased police presence. This example illustrates why distinguishing causation from correlation is crucial in data analysis, and that stating that two variables are correlated does not imply causation.

First, let’s clarify the distinction between causation and correlation. Correlation refers to a relationship between two variables, but it does not imply that one causes the other. Just because two events occur together does not mean that one directly influences the other. To establish causation, we need methods that separate the true effect of an intervention from other influencing factors.

Statisticians, medical researchers and economists have ingeniously come up with several techniques that allow us to separate correlation and causation. 

In medicine, the gold standard for researchers is the use of Randomised Controlled Trials (RCTs). Imagine a group of 100 people, each with a set of characteristics, such as gender, age, political views, health status, university degree, etc. RCTs randomly assign each individual to one of two groups. Consequently, each group of 50 individuals should, on average, have similar ages, gender distribution, and baseline health. Researchers then examine both groups simultaneously while changing only one factor. This could involve instructing one group to take a specific medicine or asking individuals to drink an additional cup of coffee each morning. This results in two statistically similar groups differing in only one key aspect. Therefore, if the characteristics of one group change while those of the other do not, we can reasonably conclude that the change caused the difference between the groups.

This is great for examining the effectiveness of medicine, especially when you give one group a placebo, but how would we research the causation behind the police rate and crime example? Surely it would be unwise and perhaps unethical to randomise how many police officers are present in each city? And because not all cities are the same, the conditions for RCTs would not hold. 

Instead, we use more complex techniques like Instrumental Variables (IV) to overcome those limitations.

A famous experiment using IV to explain police levels and crime was published by Steven Levitt (1997). Levitt used the timings of mayoral and gubernatorial elections (the election of a governor) as an instrument for changes in police hiring. Around election time, mayors and governors have incentives to look “tough on crime.” This can lead to politically motivated increases in police hiring before an election. Crucially, hiring is not caused by current crime rates but by the electoral calendar. So, by using the timing of elections to predict an increase in police, we can use those values to estimate the effect on crime. What he found was that more police officers reduce violent and property crime, with a 10% increase in police officers reducing violent crime by roughly 5%. 

Levitt’s paper is a clever application of IV to get around the endogeneity problem and takes correlation one step further into causation, through the use of exogenous election timing. However, these methods are not without limitations. IV analysis, for instance, hinges on finding a valid instrument—something that affects the independent variable (e.g., police numbers) but has no direct effect on the outcome (e.g., crime) other than through that variable. Finding such instruments can be extremely challenging, and weak or invalid instruments can lead to biased or misleading results. 

Despite these challenges, careful causal inference allows researchers to better understand the true drivers behind complicated relationships. In a world where influencers, media outlets, and even professionals often mistake correlation for causation, developing a critical understanding of these concepts is an essential skill required to navigate through the data, as well as help drive impactful change in society through exploring the true relationships behind different phenomena.

Written by George Chant

Related article: Correlation between HDI and mortality rate

REFERENCE

Steven D. Levitt (1997). “Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime”. American Economic Review 87.3, pp. 270–290