Working Paper: NBER ID: w22167
Authors: Abhijit Banerjee; Sylvain Chassang; Erik Snowberg
Abstract: A modern, decision-theoretic framework can help clarify important practical questions of experimental design. Building on our recent work, this chapter begins by summarizing our framework for understanding the goals of experimenters, and applying this to re-randomization. We then use this framework to shed light on questions related to experimental registries, pre-analysis plans, and most importantly, external validity. Our framework implies that even when large samples can be collected, external decision-making remains inherently subjective. We embrace this conclusion, and argue that in order to improve external validity, experimental research needs to create a space for structured speculation.
Keywords: Decision Theory; Experimental Design; External Validity
JEL Codes: B23; C9; O1
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| rerandomization (C90) | subjective value of experiments (C90) |
| rerandomization (C90) | robustness of policy inferences (J18) |
| external policy advice (F55) | experimenter's posterior beliefs about treatment effects (C90) |
| evidence from one environment (C91) | audience's beliefs about another environment (D83) |
| randomized experiments (C90) | prior-free inference (D84) |
| randomized experiments (C90) | robustness of policy inferences (J18) |