Separate ratings can tell a buyer whether a product is comfortable, durable or easy to use. They can also change the overall score the reviewer gives.
The operator must decide whether the extra information is worth the form’s burden and whether the headline score still means the same thing. Higher stars cannot answer both questions.
🔎 What research says
Attribute detail can improve matching while changing the overall measure. Test useful dimensions, retain an unchanged baseline, and check whether the scores remain informative about customer outcomes.
🗺️ In this guide
1. Choose attributes that help buyers find a fit
📈 Recommendation
Pilot separate ratings where a single overall score hides a meaningful product trade-off. Choose attributes buyers can judge after real use and that future buyers need to compare.
For a backpack, test fit, carrying comfort and durability if those are recurring decision questions. Keep the reviewed size and use situation attached.
Avoid decorative dimensions that simply repeat overall satisfaction. Check whether buyers understand what each rating measures.
Judge the pilot by better product matching and relevant returns, alongside review completion. More rating fields create work for reviewers.
🎓 Findings
Chen, Hong and Liu (2018) combined a restaurant-platform change with two randomized experiments. Their primary abstract reports that multidimensional ratings helped consumers find options that fit their preferences and increased choice confidence.
Verified effect sizes were not recovered from the available primary abstract. This gives a reason to test useful attribute information, not a numerical sales target. Higher overall scores after the platform change also cannot by themselves distinguish better matching from a changed measurement process.
🧠 Why it works
A buyer may care about comfort while another cares about durability. Separate attributes can reveal that trade-off instead of asking one overall score to represent both people.
✋ Limitations
The core setting was restaurants and experimental decisions. The backpack dimensions are our proposed application. Subsequent papers challenge the interpretation of higher overall ratings; more detail should earn its place through actual matching performance.
2. Preserve a comparable overall-rating baseline
📈 Recommendation
Treat a review-form change as a possible change in the measurement itself. Keep an unchanged overall-only comparison while piloting attribute questions, and mark the transition in internal score reporting.
Randomly assign eligible reviewers to the existing and proposed forms. Keep timing, invitation and purchased-product mix comparable.
Record form version with every rating. Compare score distributions, missing answers and completion, not just the new mean.
Do not attribute a higher star average to product improvement until a stable measure or buyer outcome supports that interpretation.
🎓 Findings
In Schneider and colleagues’ experiment (2021), 166 analyzed participants rated an imagined restaurant experience. Asking for four attribute ratings before the overall rating produced an average of 6.75 versus 6.23 on a nine-star scale: 0.52 stars higher. The experience described to people had not improved.
The paper also found that which attributes were rated could move the overall score in either direction. Asking for overall first was not a guaranteed cure: one later condition still showed a small, uncertain difference. The nine-star result cannot be transferred as a half-star change on a five-star store scale.
🧠 Why it works
The form directs attention to parts of the experience. A changed score can therefore reflect how the customer was asked, making an unqualified before-and-after comparison misleading.
✋ Limitations
This numerical contrast came from a fictitious restaurant, not a physical-product purchase. Attribute selection and the ability to revisit earlier answers matter. An unchanged randomized baseline is our measurement safeguard, not a formula proven to remove every bias.
3. Check whether higher ratings remain informative
📈 Recommendation
Judge the new form by whether its ratings still predict a useful customer outcome. Higher scores alone do not establish more satisfied customers.
For a chair, compare overall ratings with later willingness to recommend, support contacts and return reasons. Keep those outcomes separate and recognize that returns also depend on policy.
Check customers reporting a poor experience separately from those reporting a good one. Examine whether specific low subratings disappear inside a higher headline score.
Keep attributes that improve useful diagnosis without making the overall score harder to interpret. Use paid repeat behavior when enough time has passed.
🎓 Findings
Mehr and Simmons (2024) randomly assigned 1,991 participants to rate a subpar imagined restaurant experience overall only or with attributes. Overall ratings were 3.43 versus 2.70 on a ten-star scale, a 0.73-point rise with attributes. Yet revisit/recommend intentions were almost identical: 1.70 versus 1.71 on a nine-point scale, with no clear statistical difference.
The link between the overall score and those intentions was weaker with attributes: correlation 0.56 versus 0.67. A correlation describes how closely two measures move together; this is not a percentage loss of accuracy. The source’s good-experience tests did not show the same clear rating increase.
🧠 Why it works
A customer may record a failure in one subrating and feel less need to repeat it in the overall score. That explanation fits the paper’s experiments, but the operator still needs to check whether the resulting headline helps future buyers.
✋ Limitations
These were intentions after an imagined restaurant, not actual revisits or paid repeat orders. An unclear difference does not prove exact equality. The result does not say that attribute information is useless; it questions judging the form by a higher overall rating.
💡 The takeaway
Use attribute ratings for better matching, preserve score comparability, and judge the form by useful customer outcomes.
Pilot a few decision-relevant attributes beside an unchanged overall-only form. Record the form version, check poor experiences separately, and compare the ratings with later customer outcomes before treating a higher average as progress.
📚 Sources and study types
Mehr, K. S., & Simmons, J. P. (2024). How Does Rating Specific Features of an Experience Alter Consumers’ Overall Evaluations of That Experience? Journal of Consumer Research,51(4),739–760. DOI. Preregistered rating-elicitation experiments, including hypothetical, remembered and real experiences; Study 10 uses intention outcomes.
Chen, P.-Y., Hong, Y., & Liu, Y. (2018;online2017). The Value of Multidimensional Rating Systems: Evidence from a Natural Experiment and Randomized Experiments. Management Science,64(10),4629–4647. DOI. Restaurant-platform natural experiment and randomized matching experiments; fulltext magnitude unavailable.
Schneider, C., Weinmann, M., Mohr, P. N. C., & vom Brocke, J. (2021;online2020). When the Stars Shine Too Bright: The Influence of Multidimensional Ratings on Online Consumer Ratings. Management Science,67(6),3871–3898. DOI. Controlled and field rating experiments; the quoted contrast is a fictitious restaurant on nine stars.
