Working Paper: CEPR ID: DP13015
Authors: Erik Snowberg; Leeat Yariv
Abstract: We leverage a large-scale incentivized survey eliciting behaviors from (almost) an entire university student population, a representative sample of the U.S. population, and Amazon Mechanical Turk (MTurk) to address concerns about the external validity of experiments with student participants. Behavior in the student population offers bounds on behaviors in other populations, and correlations between behaviors are largely similar across samples. Furthermore, non-student samples exhibit higher measurement error. Adding historical lab participation data, we find a small set of attributes over which lab participants differ from non-lab participants. Using an additional set of lab experiments, we see no evidence of observer effects.
Keywords: Lab Selection; External Validity; Experiments
JEL Codes: B41; C80; C90
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| university students (I23) | behavioral differences from representative populations (D91) |
| CCS students (D29) | higher cognitive skills and strategic sophistication compared to MTurk and representative samples (C91) |
| self-selected lab participants (C91) | slight differences in generosity and risk aversion compared to the broader student population (D29) |
| no significant observer effects (C90) | participants' behaviors in the lab are similar to those in the CCS survey (C92) |