Working Paper: CEPR ID: DP8681
Authors: Javier Ruiz-Castillo
Abstract: This paper reviews a number of recent contributions that demonstrate that a blend of welfare economics and statistical analysis is useful in the evaluation of the citations received by scientific papers in the periodical literature. The paper begins by clarifying the role of citation analysis in the evaluation of research. Next, a summary of results about the citation distributions? basic features at different aggregation levels is offered. These results indicate that citation distributions share the same broad shape, are highly skewed, and are often crowned by a power law. In light of this evidence, a novel methodology for the evaluation of research units is illustrated by comparing the high- and low-citation impact achieved by the U.S., the European Union, and the rest of the world in 22 scientific fields. However, contrary to recent claims, it is shown that mean normalization at the sub-field level does not lead to a universal distribution. Nevertheless, among other topics subject to ongoing research, it appears that this lack of universality does not preclude sensible normalization procedures to compare the citation impact of articles in different scientific fields.
Keywords: citation analysis; power laws; research performance; welfare economics
JEL Codes: A11; A12; D60; I32
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| citation distributions across scientific fields are highly skewed (C46) | a small number of articles receive a disproportionate amount of citations (A14) |
| mean normalization at the subfield level does not produce a universal distribution (D39) | citation distributions share fundamental characteristics but exhibit significant differences (C46) |
| using a single statistic like the mean citation rate (MCR) (A14) | may not adequately capture the distribution's characteristics (C46) |
| high and low-impact measures (E01) | better summarize citation distributions (A14) |