# Test shows Claude models pick different Shopify products

Digest AI · Marketing & Small Business · published 2026-11-02T13:00:14Z

Canonical: https://digestai.news/story/test-shows-claude-models-pick-different-shopify-products

## Summary

A Search Engine Journal test reveals that three Claude models—Haiku 4.5, Sonnet 5, and Opus 5—select different products from the same Shopify catalog list when given identical shopping requests. The author tested 1,980 picks across six product categories using Shop MCP, Shopify’s keyless API for AI agents launched on Oct. 13, 2026. The results showed no agreement among the models on which product to buy, indicating that the specific AI model powering a shopping agent significantly influences purchase decisions.

Haiku and Sonnet consistently favored products with the highest review counts, while Opus 5 preferred "safe," general-purpose options, citing safety in 222 of 300 reasons. A notable discrepancy emerged with a water bottle listed in Shopify’s catalog as having 447,221 reviews, a figure not visible on the retailer’s product page, which showed only 6,537 reviews. Both Haiku and Sonnet relied on this inflated catalog number to make their selections, whereas Opus ignored it. Additionally, moving a product from the first to the last position in the list significantly reduced its selection rate for all models, despite none mentioning position in their reasoning.

The study highlights that agentic commerce outcomes depend heavily on the underlying language model’s biases and the data quality of the catalog it accesses. As companies build shopping agents, the choice of model and the accuracy of product metadata will determine which stores receive customer orders.

## Key points

- Three Claude models picked different winners from the same 20-product Shopify list in 1,980 tests.
- Haiku and Sonnet favored high-review items; Opus 5 chose 'safe' general-purpose products in 222 of 300 picks.
- Shopify’s catalog listed a water bottle with 447,221 reviews, a figure not shown on the product page.

## Why it matters

This reveals that AI shopping agents introduce model-specific bias into commerce. Retailers must ensure catalog data accuracy, as models rely on provided metadata, and consumers may unknowingly buy based on an AI’s default preferences rather than their own needs.

## Sources

1. [3 AI Models Shopped Shopify’s Catalog – Each Picked A Different Winner via @sejournal, @slobodanmanic](https://searchenginejournal.com/3-ai-models-shopped-shopifys-catalog-each-picked-a-different-winner/594298) (Search Engine Journal, 2026-11-02)

## Cite

Digest AI, "Test shows Claude models pick different Shopify products", 2 November 2026, https://digestai.news/story/test-shows-claude-models-pick-different-shopify-products

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/test-shows-claude-models-pick-different-shopify-products.json