Working Paper: NBER ID: w26389
Authors: Bharat K. Chandar; Ali Hortasu; John A. List; Ian Muir; Jeffrey M. Wooldridge
Abstract: Field experiments conducted with the village, city, state, region, or even country as the unit of randomization are becoming commonplace in the social sciences. While convenient, subsequent data analysis may be complicated by the constraint on the number of clusters in treatment and control. Through a battery of Monte Carlo simulations, we examine best practices for estimating unit-level treatment effects in cluster-randomized field experiments, particularly in settings that generate short panel data. In most settings we consider, unit-level estimation with unit fixed effects and cluster-level estimation weighted by the number of units per cluster tend to be robust to potentially problematic features in the data while giving greater statistical power. Using insights from our analysis, we evaluate the effect of a unique field experiment: a nationwide tipping field experiment across markets on the Uber app. Beyond the import of showing how tipping affects aggregate market outcomes, we provide several insights on aspects of generating and analyzing cluster-randomized experimental data when there are constraints on the number of experimental units in treatment and control.
Keywords: Cluster-Randomized Field Experiments; Panel Data; Tipping; Uber Marketplace
JEL Codes: C23; C33; C5; C9; C91; C92; C93; D47
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| unit-level estimation with unit fixed effects (C51) | higher power than unweighted cluster-level estimates (C38) |
| more treated clusters than control clusters (C38) | enhance precision of treatment effect estimates (C22) |
| introducing tipping (Y20) | short-run increase in labor supply (J20) |
| introducing tipping (Y20) | decrease in demand (D12) |
| introducing tipping (Y20) | decrease in hourly earnings for drivers (J31) |