Working Paper: NBER ID: w16016
Authors: John DiNardo; David S. Lee
Abstract: This chapter provides a selective review of some contemporary approaches to program evaluation. One motivation for our review is the recent emergence and increasing use of a particular kind of "program" in applied microeconomic research, the so-called Regression Discontinuity (RD) Design of Thistlethwaite and Campbell (1960). We organize our discussion of these various research designs by how they secure internal validity: in this view, the RD design can been seen as a close "cousin" of the randomized experiment. An important distinction which emerges from our discussion of "heterogeneous treatment effects" is between ex post (descriptive) and ex ante (predictive) evaluations; these two types of evaluations have distinct, but complementary goals. A second important distinction we make is between statistical statements that are descriptions of our knowledge of the program assignment process and statistical statements that are structural assumptions about individual behavior. Using these distinctions, we examine some commonly employed evaluation strategies, and assess them with a common set of criteria for "internal validity", the foremost goal of an ex post evaluation. In some cases, we also provide some concrete illustrations of how internally valid causal estimates can be supplemented with specific structural assumptions to address "external validity": the estimate from an internally valid "experimental" estimate can be viewed as a "leading term" in an extrapolation for a parameter of interest in an ex ante evaluation.
Keywords: program evaluation; regression discontinuity; causal inference; internal validity; ex post evaluation; ex ante evaluation
JEL Codes: C10; C50; C52; H00; I00; J00; J24
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| ex post evaluations (O22) | high internal validity (C90) |
| ex post evaluations (O22) | causal phenomena (C22) |
| ex ante evaluations (H43) | challenges in establishing external validity (C90) |
| ex post evaluations (O22) | inform ex ante predictions (D84) |
| average treatment effect (ATE) (C22) | extrapolating impacts of full-scale program implementations (C93) |
| randomized experiments (C90) | robust internal validity (C90) |
| randomized experiments (C90) | predict outcomes accurately when scaled up (C52) |
| RD design (O32) | credible causal inferences (C20) |
| RD design (O32) | control for confounding variables (C90) |