Two Key Techniques for Quantifying the Robustness of Causal Inferences

Get hands-on experience with quantifying the sensitivity of a causal inference using two specialized techniques – Robustness of Inference to Replacement (RIR) and Impact Threshold for a Confounding Variable (ITCV).

The phrase “But have you controlled for …” is fundamental to social science, but can also create a quandary. Even after controlling for the most likely alternative explanations for an inferred effect, there may be some alternative explanation(s) that cannot be ruled out with observed data. Generally, the first response is to develop the best models that maximally leverage the available data. After that, sensitivity analyses can inform discourse about an inference by quantifying the unobserved conditions necessary to change the inference.

In this course, you will learn how to generate statements such as “An omitted variable would have to be correlated at ___ with the predictor of interest and with the outcome to change the inference.” Or “To invalidate the inference, __% of the data would have to be replaced with counterfactual cases for which the treatment had no effect.” Because these statements express sensitivity in terms of correlations or cases they have wide accessibility.

Rooted in the foundations of the general linear model and potential outcomes, the techniques can be adapted to a range of analyses, including logistic regression, propensity-based approaches, and multilevel models. As a result, they can broadly facilitate discourse about inferences among researchers who seek to make an inference, challengers of that inference, policymakers, and clinicians.

In this seminar, you will:

1. Apply and understand techniques for quantifying the robustness of causal inferences.
   1. Comparing evidence to a threshold for inference.
   2. Understanding internal and external validity.
2. Conduct sensitivity analyses in R or the on-line app http://konfound-it.com (Stata and Excel also available).
3. Develop a deeper understanding of regression and the counterfactual as well as how threats to internal and external validity compare against the strength of evidence.
4. Apply sensitivity analysis to a specific problem of interest that may require extensions or adaptations.