Working Paper: NBER ID: w19508
Authors: Katherine B. Coffman; Lucas C. Coffman; Keith M. Marzilli Ericson
Abstract: Measuring sexual orientation, behavior, and related opinions is difficult because responses are biased towards socially acceptable answers. We test whether measurements are biased even when responses are private and anonymous and use our results to identify sexuality-related norms and how they vary. We run an experiment on 2,516 U.S. participants. Participants were randomly assigned to either a "best practices method" that was computer-based and provides privacy and anonymity, or to a "veiled elicitation method" that further conceals individual responses. Answers in the veiled method preclude inference about any particular individual, but can be used to accurately estimate statistics about the population. Comparing the two methods shows sexuality-related questions receive biased responses even under current best practices, and, for many questions, the bias is substantial. The veiled method increased self-reports of non-heterosexual identity by 65% (p<0.05) and same-sex sexual experiences by 59% (p<0.01). The veiled method also increased the rates of anti-gay sentiment. Respondents were 67% more likely to express disapproval of an openly gay manager at work (p<0.01) and 71% more likely to say it is okay to discriminate against lesbian, gay, or bisexual individuals (p<0.01). The results show non-heterosexuality and anti-gay sentiment are substantially underestimated in existing surveys, and the privacy afforded by current best practices is not always sufficient to eliminate bias. Finally, our results identify two social norms: it is perceived as socially undesirable both to be open about being gay, and to be unaccepting of gay individuals.
Keywords: LGBT; Antigay sentiment; Survey methods; Social desirability bias
JEL Codes: C90; D10; J10
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| direct report method (C59) | reported nonheterosexual identity (J79) |
| veiled report method (Y30) | reported nonheterosexual identity (J79) |
| direct report method (C59) | reports of same-sex sexual experiences (J16) |
| veiled report method (Y30) | reports of same-sex sexual experiences (J16) |
| direct report method (C59) | antigay sentiment (J71) |
| veiled report method (Y30) | antigay sentiment (J71) |
| veiled report method (Y30) | belief that discrimination against LGBT individuals is acceptable (J71) |
| direct report method (C59) | belief that discrimination against LGBT individuals is acceptable (J71) |