The phrase “But have you controlled for …” is fundamental to social science, but can also create a quandary. Even after controlling for the most likely alternative explanations for an inferred effect, there may be some alternative explanation(s) that cannot be ruled out with observed data.  Generally, the first response is to develop the best models that maximally leverage the available data. After that, sensitivity analyses can inform discourse about an inference by quantifying the unobserved conditions necessary to change the inference.

This course provides widely accessible ways, such as correlations or cases, to quantify the sensitivity of an inference. Specifically, in this course you will learn how to generate statements such as “An omitted variable would have to be correlated at ___ with the predictor of interest and with the outcome to change the inference.” Or “To invalidate the inference, __% of the data would have to be replaced with counterfactual cases for which the treatment had no effect.”

Rooted in the foundations of the general linear model and potential outcomes, these techniques can be adapted to a range of analyses, including logistic regression, propensity-based approaches, and multilevel models. As a result, they can broadly facilitate discourse among researchers who seek to make an inference, challengers of that inference, as well as policymakers and clinicians.