Working Paper: NBER ID: w26250
Authors: Fiona Burlig; Louis Preonas; Matt Woerman
Abstract: How should researchers design panel data experiments? We analytically derive the variance of panel estimators, informing power calculations in panel data settings. We generalize Frison and Pocock (1992) to fully arbitrary error structures, thereby extending McKenzie (2012) to allow for non-constant serial correlation. Using Monte Carlo simulations and real world panel data, we demonstrate that failing to account for arbitrary serial correlation ex ante yields experiments that are incorrectly powered under proper inference. By contrast, our “serial-correlation-robust” power calculations achieve correctly powered experiments in both simulated and real data. We discuss the implications of these results, and introduce a new software package to facilitate proper power calculations in practice.
Keywords: panel data; experimental design; power calculations; serial correlation
JEL Codes: B4; C23; C9; O1; Q4
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Failing to account for serial correlation (C22) | Incorrect power estimations (C51) |
| Incorrect power estimations (C51) | Biased results of randomized controlled trials (RCTs) (C90) |
| Failing to account for serial correlation ex ante (C22) | Overpowered or underpowered experiments (C90) |
| Serial-correlation-robust (SCR) formula (C29) | Achieves desired statistical power (C52) |
| SCR formula (C20) | Properly powered experiments (C90) |
| Incorporating serial correlation into power calculations (C22) | Avoid type I errors (C52) |