Working Paper: NBER ID: w29756
Authors: Frederico Finan; Demian Pouzo
Abstract: This paper presents a framework for how to incorporate prior sources of information into the design of a sequential experiment. These sources can include previous experiments, expert opinions, or the experimenter's own introspection. We formalize this problem using a multi-prior Bayesian approach that maps each source to a Bayesian model. These models are aggregated according to their associated posterior probabilities. We evaluate a broad of policy rules according to three criteria: whether the experimenter learns the parameters of the payoff distributions, the probability that the experimenter chooses the wrong treatment when deciding to stop the experiment, and the average rewards. We show that our framework exhibits several nice finite sample properties, including robustness to any source that is not externally valid.
Keywords: Bayesian learning; External validity; Sequential experiments; Policy design
JEL Codes: C11; C50; C90; O12
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| amount of experimentation (C90) | learning rate of the policymaker (D78) |
| length of the experiment (C90) | probability of making a mistake (C52) |
| degree of experimentation (C99) | probability of making a mistake (C52) |
| treatment effect size (C21) | probability of making a mistake (C52) |
| effective treatment assignment (C90) | average payoff of the policymaker (D72) |