Working Paper: NBER ID: w27554
Authors: Martin Ravallion
Abstract: The rising popularity of randomized controlled trials (RCTs) in development applications has come with continuing debates about the merits of this approach. The paper takes stock of the issues. It argues that an unconditional preference for RCTs is questionable on three main counts. First, the case for such a preference is unclear on a priori grounds. For example, with a given budget, even a biased observational study can come closer to the truth than a costly RCT. Second, the ethical objections to RCTs have not been properly addressed by advocates. Third, there is a risk of distorting the evidence-base for informing policymaking, given that an insistence on RCTs generates selection bias in what gets evaluated. Going forward, pressing knowledge gaps should drive the questions asked and how they are answered, not the methodological preferences of some researchers. The gold standard is the best method for the question at hand.
Keywords: randomized controlled trials; impact evaluation; development economics
JEL Codes: B41; C93; O22
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| method of evaluation (RCT vs. observational) (C90) | accuracy of the impact estimates (C13) |
| ethical framework guiding evaluations (A13) | outcomes observed in RCTs (C90) |
| insistence on RCTs (C90) | selection bias in what gets evaluated (C52) |
| methodological bias favoring RCTs (C90) | overall evidence base for policymaking (D78) |