Working Paper: NBER ID: w23867
Authors: Abhijit Banerjee; Sylvain Chassang; Sergio Montero; Erik Snowberg
Abstract: This paper proposes a decision-theoretic framework for experiment design. We model experimenters as ambiguity-averse decision-makers, who make trade-offs between subjective expected performance and robustness. This framework accounts for experimenters' preference for randomization, and clarifies the circumstances in which randomization is optimal: when the available sample size is large enough or robustness is an important concern. We illustrate the practical value of such a framework by studying the issue of rerandomization. Rerandomization creates a trade-off between subjective performance and robustness. However, robustness loss grows very slowly with the number of times one randomizes. This argues for rerandomizing in most environments.
Keywords: experiment design; randomization; rerandomization; ambiguity aversion; decision theory
JEL Codes: C90; D81
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| sample size (C83) | robustness of RCTs (C90) |
| deterministic experiments (C90) | subjective expected utility (D81) |
| rerandomization (C90) | covariate balance (C10) |
| rerandomization (C90) | robustness (L15) |
| number of rerandomizations (C90) | impact on robustness (L15) |
| randomized controlled trials (RCTs) (C90) | robust prior-free inference (C11) |