AI agents shopping: big promise, modest results

Shopify has opened a directory for AI agents, OpenAI is selling ads in ChatGPT, and Polish stores are almost not ready for this at all. We checked how much agentic commerce actually delivers today and what a store can do so as not to overpay for a fad.

Agentic commerce (in English, agentic commerce) means shopping in which software based on a language model searches for, compares, and buys a product for the customer on its own. The industry talks about it as the next big breakthrough. We compared three fresh data sources: an original test of AI models on the Shopify catalog, an academic paper on ads in ChatGPT, and a Polish audit of store readiness. For now, the data paints more a picture of an experiment than of a proven sales channel.

A billion products opened to agents

On October 13, 2026, Shopify launched Shop MCP. MCP (Model Context Protocol) is a standard for connecting AI agents with external services. Through one such connection, an agent can search a catalog of more than a billion products, build a cart, and pay via Shop Pay. For search alone, it does not even need an API key, that is, individual access issued by the platform.

According to industry analyses, stores on Shopify become “agent-ready” by default, and the Shop channel cannot be completely turned off. For the seller, this means agents will see their offer anyway. The open question is whether they will choose anything from it.

The same cart, three different choices

Slobodan Manic, author of the test published in Search Engine Journal, gave three Claude models (Haiku 4.5, Sonnet 5, and Opus 5) the same lists of 20 products from the Shopify catalog and the same short instruction: choose one product for the customer. In total, he collected 1,980 choices. Across six prompts, from a backpack to coffee, the models did not agree even once on the most frequently chosen product.

At the same time, each model was consistent with itself: Sonnet’s favorite product accounted for an average of 94 percent of its choices, Opus 89 percent, and Haiku 67 percent. Haiku and Sonnet leaned toward products with the highest number of reviews. In 222 of 300 explanations, Opus called its choice “safe” and reached for universal versions, for example medium-roast coffee (48 times out of 50).

The most instructive case is the nobl brand water bottle. The catalog listed it as having 5 stars from 447,221 reviews, while the product page showed 6,537, about 68 times fewer. Haiku and Sonnet chose it citing precisely that number, even though they had no way to verify it. In a similar way, they repeated phrases from descriptions: the phrase “clinically proven” appeared in 63 explanations, 61 times directly from the product description.

The test also showed the impact of factors the customer will not learn about. When, in eight cosmetics prompts, a product was moved from first place in the catalog to last, the number of times it was chosen dropped, for example for Haiku from 28 to 11 out of 80 trials; however, the effect was clear only for some prompts, including sunscreen. None of the 480 explanations mentioned position on the list. On the other hand, in the sunscreen test, it was enough to add a budget (under $25) and skin type to the prompt for the models to abandon the product ranked highest by the catalog at $48 and choose a cheaper one in 298 out of 300 trials.

This is a limited test: only models from one company, without the instructions shopping apps add, and without real purchases. Still, the editorial conclusion is clear. Which store gets the order depends in part on which model powers the customer’s agent. The store does not choose that and cannot monitor it the way it monitors its positions in Google.

For now, advertising in ChatGPT loses to search

The second promise is advertising in AI assistants. Maximilian Kaiser (Vienna University of Economics and Business) and Christian Schulze (Frankfurt School of Finance & Management) analyzed data from 102 advertiser websites with full sales tracking. In US traffic, a visit from a ChatGPT ad generated $0.89 in revenue, while a visit from paid Google advertising generated $2.45. Conversion was 0.37 percent and 1.17 percent respectively.

Among 87 companies that bought both types of advertising, 84 earned less per visit from ChatGPT. As many as 73 recorded no sales from it at all despite a total of 18,726 visits. One advertiser accounted for 66 percent of all purchases from this channel. The authors calculated the so-called parity price: at a return comparable to search, a click in ChatGPT would be worth about $0.26. Meanwhile, the largest publicly known account was paying $1.72 per click, and OpenAI recommends starting campaigns with a maximum bid of $3-5 per click.

These figures should be approached cautiously. The paper is still under review, the data ends on August 22, comes from Grips Intelligence, a company for which Kaiser is an advisor, and the advertisers themselves decided to buy the ads. There is also more favorable data: OpenAI reported a threefold return on ad spend for one store over 28 days, and according to DoubleVerify, customer acquisition cost at Weight Watchers was 15.3 percent lower than the assumed benchmark. However, these are isolated cases and different metrics that cannot be directly compared with the study data.

Polish stores will not let an agent reach checkout

On the Polish side, the point of reference is Digitree Group’s “Agentic Readiness Index 2026,” issued under the honorary patronage of IAB Polska. It is a corporate report, but based on an audit of technical infrastructure rather than declarations. In August 2026, 1,665 Polish stores were checked: 97.7 percent received the lowest grades, D or F, and only 28 received an A. The average score was 21.2 points out of 100, compared with 29.1 points in a sample of 395 European stores.

Polish stores usually let AI bots in, but 94 percent do not provide prices and availability in a machine-readable format. AI-assisted cart handling is enabled by 38 stores, and 37 have a UCP manifest (Universal Commerce Protocol, a standard describing purchases by agents).

The data shows that an agent will usually find a product in a Polish store, but will not complete the purchase for the customer.

However, these shortcomings need to be viewed against the scale of the phenomenon. According to Boston Consulting Group, referrals from external agents accounted for less than 1 percent of visits among European retailers at the end of 2025; the firm estimates this could rise to as much as 5 percent in 2026-2027. So the problem is real, but the traffic stores are losing because of it today is small.

What a store can do without overpaying

Our conclusion: agentic commerce does not yet work like a mature sales channel, and advertising in ChatGPT is not yet justified by the results. That does not mean it can be ignored. Most of the changes that prepare a store for agents also organize data for search engines and comparison engines, so they are not an investment in a fad alone.

• Check what the catalog or product feed shows the machine: rating, number of reviews, description, and features.
• Make prices and availability available in structured data, for example in JSON-LD format (a product data notation readable by search engines and agents).
• Write descriptions using the words customers use to describe their needs, not slogans.
• Remove claims from descriptions that you cannot defend, because the agent will repeat them to the customer as fact.
• Run your category through several models together with competitors’ offers and see what they choose.

Before buying ads in AI assistants, calculate the profitability threshold using your own data, not the platform’s promises.