Working Paper: NBER ID: w25504
Authors: Charles F. Manski
Abstract: Statisticians have proposed meta-analysis to combine the findings of multiple studies of health risks or treatment response. The standard practice is to compute a weighted-average of the estimates. Yet it is not clear how to interpret a weighted average of estimates reported in disparate studies. Meta-analyses often answer this question through the lens of a random-effects model, which interprets a weighted average of estimates as an estimate of a mean parameter across a hypothetical population of studies. The relevance to medical decision making is obscure. Decision-centered research should aim to inform risk assessment and treatment for populations of patients, not populations of studies. This paper lays out principles for decision-centered meta-analysis. One first specifies a prediction of interest and next examines what each available study credibly reveals. Such analysis typically yields a set-valued prediction rather than a point prediction. Thus, one uses each study to conclude that a probability of disease, or mean treatment response, lies within a range of possibilities. Finally, one combines the available studies by computing the intersection of the set-valued predictions that they yield. To demonstrate decision-centered meta-analysis, the paper considers assessment of the effect of anti-hypertensive drugs on blood pressure.
Keywords: No keywords provided
JEL Codes: C18; I1
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| standard practice of computing a weighted average of estimates from disparate studies (C51) | obscures the relevance of these averages to actual clinical decisions (D87) |
| divergence between study populations and patient populations (I14) | misinterpretations of the findings (Y50) |
| decision-centered meta-analysis should replace weighted averages with the intersection of set-valued predictions (D79) | provides a more accurate representation of the uncertainties and variabilities in patient outcomes (C53) |
| traditional methods inadequately address statistical imprecision and identification problems (C52) | misinterpretations of the findings (Y50) |
| intersection of predictions from multiple studies (C52) | yields more clinically relevant information than a simple average of estimates (C13) |