Working Paper: NBER ID: w21517
Authors: Ben Gillen; Erik Snowberg; Leeat Yariv
Abstract: Measurement error is ubiquitous in experimental work. It leads to imperfect statistical controls, attenuated estimated effects of elicited behaviors, and biased correlations between characteristics. We develop simple statistical techniques for dealing with experimental measurement error. These techniques are applied to data from the Caltech Cohort Study, which conducts repeated incentivized surveys of the Caltech student body. We illustrate the impact of measurement error by replicating three classic experiments, and showing that results change substantially when measurement error is taken into account. Collectively, these results show that failing to properly account for measurement error may cause a field-wide bias: it may lead scholars to identify "new" effects and phenomena that are actually similar to those previously documented.
Keywords: Measurement Error; Experimental Economics; Behavioral Proxies
JEL Codes: C81; C9; D8; J71
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Measurement Error (C83) | Biased Correlations (C46) |
| Measurement Error (C83) | Attenuated Estimated Effects (C51) |
| Measurement Error (C83) | Validity of Experimental Results (C90) |
| Measurement Error (C83) | Misidentified Effects (D91) |
| Measurement Error (C83) | Field-wide Bias (J70) |
| Risk Attitudes (X) (D81) | Gambling Behavior (D) (D91) |
| Gambling Behavior (D) (D91) | Participation in Dangerous Sports (Y) (Z29) |