Working Paper: NBER ID: w25320
Authors: Christopher J. Ruhm
Abstract: This paper examines potential tradeoffs between research methods in answering important questions versus providing more cleanly identified estimates on problems that are potentially of lesser interest. The strengths and limitations of experimental and quasi-experimental methods are discussed and it is postulated that confidence in the results obtained may sometimes be overvalued compared to the importance of the topics addressed. The consequences of this are modeled and several suggestions are provided regarding possible steps to encourage greater focus on questions of fundamental importance.
Keywords: No keywords provided
JEL Codes: A11; B4; C50; H0; I0; J0; O0
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| excessive reliance on experimental and quasi-experimental methods (C90) | neglect of critical issues (I30) |
| prioritizing confidence in results (C12) | detract from exploration of important questions (A12) |
| small RCTs (C90) | imprecisely estimated average treatment effects (ATE) (C22) |
| clean identification (Y50) | precise estimates (C13) |
| clean identification (Y50) | limited applicability of findings (C90) |
| focus on confidence (G53) | under-investigation of crucial topics (D80) |