A shopper opens a dress they like. Your page suggests another one. That suggestion might rescue a poor match—or distract someone who was ready to buy. More clicks cannot tell you which happened.
The decision is whether alternatives help the customer choose and improve the store’s overall result. Follow purchases across the original item and its substitutes, make price trade-offs clear, and distinguish added interest from paid orders.
🔎 What research says
Evaluate the original item and its alternatives together. Test price gaps rather than assuming every related product is a useful substitute. Track what happens beyond the recommendation click, including whether attention becomes a paid purchase.
🗺️ In this guide
1. Measure the original item and its alternatives together
📈 Recommendation
When testing substitutes on a product page, measure both the original item and the products recommended beside it. A lower purchase rate for the first item can coexist with a better result across the choice set.
Keep a group on the current page and assign comparable visitors to the alternative version. Keep prices, availability and promotions comparable.
For a dress category, track purchases of the original dress, its recommended substitutes and other dresses. Add category margin and returns before deciding to roll out. This is our proposed store test.
🎓 Findings
Kumar and Hosanagar (2019) ran a nine-week apparel-store experiment comparing product pages with up to four related products against pages without recommendations. The author manuscript describes 869,764 included visitor sessions, not that many distinct people.
The published paper reports 7.5% more daily product-page views. Conditional on a product-page view, sales of that product fell 1.9%, while sales of recommended substitutes rose 9%. These are relative changes in different measures; they cannot be added or subtracted into one conversion result.
The author manuscript’s descriptive table shows $1.594 versus $1.431 in average daily sales for an original product and its recommended products together. That is about 16 cents more per focal-product set per day. It measures sales revenue, not profit, and is not a forecast for another store.
🧠 Why it works
A substitute can divert a purchase from one SKU while helping the shopper find a better match. Recommendations also give products exposure on other pages. Looking only at the first SKU misses that movement; looking only at substitute clicks misses whether anyone buys.
✋ Limitations
This was one fashion retailer and one recommendation system. The paper measured sales, not profit or long-term satisfaction. The author manuscript and published abstract agree on the headline effects, but the full final paper and supplement still need a complete comparison.
2. Test whether the price gap makes sense
📈 Recommendation
Check the difference between the price of the item being viewed and each suggested substitute. Test whether closer price ranges help customers compare, while keeping genuinely useful upgrades and cheaper alternatives available.
For a £60 dress, check why the system suggests a £150 dress. A visible difference in fabric, occasion or construction may justify the comparison; a shared colour alone may not. These prices illustrate the task, not a researched threshold.
Compare alternatives within similar price ranges with your current recommendations. Measure purchases across the choice set, margin and returns; do not improve substitute clicks at the expense of a suitable purchase.
🎓 Findings
Wan, Kumar and Li (2026; online 2025) studied recommendations at a US apparel retailer. Their published summary reports 21% lower purchase probability for a recommended product when its price differed from the product being viewed. The pattern was stronger for closely related products.
The 21% is a relative reduction, not a fall of 21 percentage points. The summary does not supply the baseline or a price-gap threshold, so it cannot tell a seller how many orders they would lose or prescribe an acceptable percentage gap.
The field study did not randomly assign price differences. The researchers also varied prices in a separate online experiment. Their 4% potential gain in total product sales came from a model of changing recommendations; it was not an observed rollout gain.
🧠 Why it works
The product already on screen provides a price reference. A very different price can make a supposedly similar item harder to interpret. That explanation fits the paper; whether a particular upgrade is confusing or useful still depends on the product and the shopper’s task.
✋ Limitations
The available publisher and university summaries do not disclose the sample, exact price-gap definition or uncertainty behind these estimates. Use the result to motivate a conditional test, not to impose a universal same-price rule or forecast a sales uplift.
3. Follow attention through to paid orders
📈 Recommendation
Track which pages send attention to an alternative and what happens next. Separate product views, bag additions and paid orders. Check nearby recommendation paths as well as the original page.
When a new handbag links to an existing one, compare discovery and bag additions with paid purchases of both products and other bags. Keep the analysis period consistent.
Check a short path through the recommendation network before expanding the feature everywhere. Record how many shoppers reach each step; use the final paid-order result and category margin for the rollout decision.
🎓 Findings
Helmers, Krishnan and Patnam (2019) analyzed Net-a-Porter product activity. They used new-product arrivals and regional differences in recommendation sets to estimate how visibility affected existing products. The main analysis covered 15,398 products.
They estimated approximately 6% more shopping-bag additions when an existing product was recommended by a new arrival. The paper’s headline calls this a sales effect, but its methods define bag additions as a sales proxy: an item put in a bag was not necessarily purchased. A 6% rise in this measure does not establish 6% more paid orders.
Effects on products further along recommendation links were smaller, and no clear effect was found three links away in the tested network. That result motivates checking where attention travels; it does not establish a universal three-click limit.
🧠 Why it works
A new item can make an older alternative visible to shoppers who would otherwise miss it. Some of that attention may travel further, but being noticed and being bought are different steps. Treating a bag addition as revenue hides the point where the purchase may fail.
✋ Limitations
The study did not randomly assign recommendation links. Its identification uses launch timing and regional comparisons, which depend on assumptions about other demand changes. The live feed sampled around half of bag and wish-list additions. The authors say the proxy effect should be interpreted as an upper bound on actual sales effects.
💡 The takeaway
Judge alternatives by the useful purchases they help shoppers make.
Start in one category with genuine substitutes. Compare the current product page with a version that shows clear alternatives, then follow the original item, recommended products, category margin and returns. Keep a change when the whole journey improves—not simply when more shoppers leave the first page.
📚 Sources and study types
Kumar, A., & Hosanagar, K. (2019). Measuring the Value of Recommendation Links on Product Demand. Information Systems Research, 30(3), 819–838. Randomized apparel-retailer experiment; original-item, substitute and combined sales. Author manuscript methods and estimates checked against the published abstract.
Wan, X. (S.), Kumar, A., & Li, X. (2026; online 2025). Is It Beneficial to Recommend Differently Priced Products? Experimental Evidence From Online Recommender Systems. Production and Operations Management, 35(5). Retailer study, separate online price experiment and modeled recommendation-policy estimate; quantitative summary verified, full-text extraction incomplete.
Helmers, C., Krishnan, P., & Patnam, M. (2019). Attention and saliency on the internet: Evidence from an online recommendation system. Journal of Economic Behavior & Organization, 161, 216–242. Product-level analysis using new arrivals and regional recommendation differences; bag additions proxy for sales, not verified paid orders.
