Working Paper: NBER ID: w22434
Authors: Brian Jacob; Jesse Rothstein
Abstract: Economists often use test scores to measure a student’s performance or an adult’s human capital. These scores reflect non-trivial decisions about how to measure and scale student achievement, with important implications for secondary analyses. For example, the scores computed in several major testing regimes, including the National Assessment of Educational Progress (NAEP), depend not only on the examinees’ responses to test items, but also on their background characteristics, including race and gender. As a consequence, if a black and white student respond identically to questions on the NAEP assessment, the reported ability for the black student will be lower than for the white student—reflecting the lower average performance of black students. This can bias many secondary analyses. Other assessments use different measurement models. This paper aims to familiarize applied economists with the construction and properties of common cognitive score measures and the implications for research using these measures.
Keywords: student ability; test scores; measurement error; educational assessment; racial bias
JEL Codes: C8; I2
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| racial background (J15) | reported ability scores (C12) |
| mismeasured ability (C52) | inflated estimates of racial wage gaps (J79) |
| measurement error in ability estimates (C20) | overstated gaps in wage analysis (J31) |
| assumption of interval properties of test scores (C12) | misinterpretation of changes in ability (D91) |