Working Paper: CEPR ID: DP18180
Authors: Christian Decker; Marco Ottaviani
Abstract: Preregistration at public research registries is considered a promising solution to the credibility crisis in science, but empirical evidence of its actual benefit is limited. Guaranteeing research integrity is especially vital in clinical research, where human lives are at stake and investigators might suffer from financial pressure. This paper analyzes the distribution of p-values from pre-approval drug trials reported to ClinicalTrials.gov, the largest registry for research studies in human volunteers, conditional on the preregistration status. The z-score density of non-preregistered trials displays a significant upward discontinuity at the salient 5% threshold for statistical significance, indicative of p-hacking or selective reporting. The density of preregistered trials appears smooth at this threshold. With caliper tests, we establish that these differences between preregistered and non-preregistered trials are robust when conditioning on sponsor fixed effects and other design features commonly indicative of research integrity, such as blinding and data monitoring committees. Our results suggest that preregistration is a credible signal for the integrity of clinical trials, as far as it can be assessed with the currently available methods to detect p-hacking.
Keywords: preregistration; clinical trials; research registries; selective reporting; p-hacking
JEL Codes: D8; C4; C9; O3
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| preregistration (K16) | smoother distribution of z-scores (C46) |
| nonpreregistered trials (C90) | upward discontinuity in z-scores (C46) |
| preregistration (K16) | credibility of reported results (C90) |
| sponsor fixed effects (C23) | control for confounding factors (C90) |
| preregistration (K16) | research integrity indicators (O36) |