Working Paper: NBER ID: w20877
Authors: Omar Alubaydli; John A. List
Abstract: A commonly held view is that laboratory experiments provide researchers with more “control” than natural field experiments, and that this advantage is to be balanced against the disadvantage that laboratory experiments are less generalizable. This paper presents a simple model that explores circumstances under which natural field experiments provide researchers with more control than laboratory experiments afford. This stems from the covertness of natural field experiments: laboratory experiments provide researchers with a high degree of control in the environment which participants agree to be experimental subjects. When participants systematically opt out of laboratory experiments, the researcher’s ability to manipulate certain variables is limited. In contrast, natural field experiments bypass the participation decision altogether and allow for a potentially more diverse participant pool within the market of interest. We show one particular case where such selection is invaluable: when treatment effects interact with participant characteristics.
Keywords: natural field experiments; laboratory experiments; control; generalizability
JEL Codes: C9; C91; C92; C93
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| laboratory experiments (C91) | greater control over the experimental environment (C90) |
| natural field experiments (C93) | greater control under certain conditions (E61) |
| participant opt-out rates (C83) | restricted ability to manipulate variables in laboratory experiments (C90) |
| natural field experiments (C93) | more representative sample (C83) |
| natural field experiments (C93) | more accurate insights into participant behavior (C91) |
| natural field experiments (C93) | enhanced reliability of findings (C90) |