Working Paper: NBER ID: w18567
Authors: Weijia Dai; Ginger Z. Jin; Jungmin Lee; Michael Luca
Abstract: Because consumer reviews leverage the wisdom of the crowd, the way in which they are aggregated is a central decision faced by platforms. We explore this "rating aggregation problem" and offer a structural approach to solving it, allowing for (1) reviewers to vary in stringency and accuracy, (2) reviewers to be influenced by existing reviews, and (3) product quality to change over time. Applying this to restaurant reviews from Yelp.com, we construct an adjusted average rating and show that even a simple algorithm can lead to large information efficiency gains relative to the arithmetic average.
Keywords: consumer ratings; Yelp; rating aggregation; review dynamics
JEL Codes: D8; L15; L86
Edges that are evidenced by causal inference methods are in orange, and the rest are in light blue.
| Cause | Effect |
|---|---|
| Adjusted Average Rating (C29) | Accurate Reflection of Restaurant Quality (L15) |
| Simple Average Rating (C29) | Systematic Bias in Restaurant Quality Reflection (L15) |
| Elite Reviewers (Y30) | Higher Precision in Observing Quality Signals (C58) |
| Non-Elite Reviewers (Y30) | Overweight Their Own Signals (G41) |
| Stringency of Reviewers (Y30) | Influenced by Reviewer History (Y30) |
| Chilling Effect (D84) | Downward Trend of Ratings Over Time (Y10) |
| Reviewer Selection Bias (C52) | Difficulty Distinguishing Quality Decline (L15) |