Working Paper: NBER ID: w14265
Authors: Melissa Clark; Jesse Rothstein; Diane Whitmore Schanzenbach
Abstract: Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in "ACT states," where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.
Keywords: college admissions; test scores; selection bias
JEL Codes: C24; I2; J24
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| test participation rates (C90) | latent achievement (D84) |
| policy reform in Illinois (K23) | test participation rates (C90) |
| observed school mean test scores (C12) | average latent scores (C29) |
| selectivity of test-takers in ACT states (H73) | latent achievement (D84) |
| selectivity of test-takers in SAT states (H73) | latent achievement (D84) |