Working Paper: NBER ID: w26080
Authors: Katherine Baicker; Theodore Svoronos
Abstract: Given the complex relationships between patients’ demographics, underlying health needs, and outcomes, establishing the causal effects of health policy and delivery interventions on health outcomes is often empirically challenging. The single interrupted time series (SITS) design has become a popular evaluation method in contexts where a randomized controlled trial is not feasible. In this paper, we formalize the structure and assumptions underlying the single ITS design and show that it is significantly more vulnerable to confounding than is often acknowledged and, as a result, can produce misleading results. We illustrate this empirically using the Oregon Health Insurance Experiment, showing that an evaluation using a single interrupted time series design instead of the randomized controlled trial would have produced large and statistically significant results of the wrong sign. We discuss the pitfalls of the SITS design, and suggest circumstances in which it is and is not likely to be reliable.
Keywords: Single Interrupted Time Series; Causal Inference; Health Policy Evaluation
JEL Codes: C1; I1; I13
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| SITS design (C90) | misleading results (C52) |
| SITS design (C90) | incorrect direction of ED utilization estimates (C51) |
| SITS design assumptions (C20) | reliability of estimates (C51) |
| SITS estimates (C13) | bias (D91) |
| Medicaid enrollment (I18) | emergency department (ED) utilization (I11) |