The Maze: ChatGPT shopping advice looks less like a shelf and more like a channel algorithm. Visibility Labs ran 1,000 product prompts 10 times with search off and 10 times with search on. Across 20,000 responses, the recommended products changed by 80.2%. The strange part sits at the top: products that appeared every time without search had only 15.8% overlap when search was enabled. The most reliable recommendations became the least portable.
Search does not just refresh answers. It rewires the candidate set. With search disabled, ChatGPT relied on its trained knowledge and returned 6.2 products per response. With search enabled, it returned 5.2 products per response and pulled from current web context. The result was not a longer list. It was a different one. Visibility Labs found only 19.8% overlap across the broad `>10%` visibility group, meaning four out of five products moved when web retrieval entered the answer.
The top of the shelf was less stable than expected. At `>60%` visibility, overlap peaked at 20.8%. Then the pattern broke: 19.2% at `>80%`, 17.2% at `>90%`, and 15.8% for products recommended 100% of the time without search. That is the uncomfortable lesson. A product can be deeply embedded in the model's non-search answer and still lose its place when the model checks fresh pages, reviews, rankings, or product content.
The methodology makes this useful for operators, not just AI watchers. Visibility Labs built prompts from transactional product keywords with shopping-result signals, minimum search volume of 1,000, and duplicate-intent filtering. Each prompt ran repeatedly so products could receive a `Visibility Score`, not a one-off mention. Product names were canonicalized to avoid counting minor naming variants as different items. It is not perfect, but it is close to how shoppers actually ask messy, category-level questions.
The commercial lever shifts from brand memory to source presence. Visibility Labs also points to its earlier 10,000-response study showing a 0.4 Pearson correlation between product mentions in cited sources and recommendation frequency. The practical read: AI shopping visibility is becoming a battle for the pages assistants retrieve. Editorial lists, comparison posts, reviews, category explainers, and product pages become shelf space. If a brand is absent there, ChatGPT may know it exists and still leave it out.
Why it matters: Ecommerce teams have spent years optimizing for Google rankings, marketplace search, and retail media auctions. AI recommendations add a new surface with a different failure mode. The answer may not punish weak products. It may punish products missing from the web sources the assistant trusts at the moment of retrieval. That makes AI visibility less like SEO as a traffic channel and more like category distribution. The shelf is dynamic. The placement rent is paid in credible mentions.
Sources: Visibility Labs
