Working Paper: CEPR ID: DP8166
Authors: Javier Ruiz-Castillo
Abstract: This paper reviews a number of recent contributions that demonstrate that a blend of welfare economics and statistical analysis is useful in the evaluation of the citations received by scientific papers in the periodical literature. The paper begins by clarifying the role of citation analysis in the evaluation of research. Next, a summary of results about the citation distributions? basic features at different aggregation levels is offered. These results indicate that citation distributions share the same broad shape, are highly skewed, and are often crowned by a power law. In light of this evidence, a novel methodology for the evaluation of research units is illustrated by comparing the high- and low-citation impact achieved by the U.S., the European Union, and the rest of the world in 22 scientific fields. However, contrary to recent claims, it is shown that mean normalization at the sub-field level does not lead to a universal distribution. Nevertheless, among other topics subject to ongoing research, it appears that this lack of universality does not preclude sensible normalization procedures to compare the citation impact of articles in different scientific fields.
Keywords: citation analysis; power laws; research performance; welfare economics
JEL Codes: 031; A11; A12; D60; I32
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Using a single statistic such as the mean citation rate (MCR) (A14) | Inadequate summary of the performance of research units (D29) |
| Skewness of citation distributions (C46) | Need for different normalization procedures across scientific fields (C90) |
| Citation distributions are highly skewed (C46) | A small number of articles receive a disproportionate amount of citations (A14) |
| The skewness of citation distributions (C46) | Disparity in citation impact between different geographical areas (R12) |