Your Store Is ‘Agent-Ready’ By Default, We Tested 122 Shopify Stores To See What That Actually Means via @sejournal, @billhunt
All Paid Media PPC News Social MediaAdvertising Video Advertising Columns Ask A PPC ExpertNEW PPC Pulse Webinar How To Create Demand You Can Actually Convert (And Prove) Join this webinar to choose a channel to test, set a starting budget, and identify results to track before investing more. Digital Marketing All Digital Marketing Analytics & Data Ecommerce Lead Generation Content Marketing Social Platforms Google YouTube Reddit LinkedIn TikTokNEW WordPress Other CMS Platforms Guide SEO vs. AI Search: Strategy Prioritization Quiz A 3-minute quiz that diagnoses your SEO and AI readiness gaps and returns a ranked action plan specific to your site. SEJ Live Webinars Rundowns PodcastNEW Ebooks All Resources Guide AI Max Invalid Traffic Doubled, Weeks Before Black Friday Invalid traffic on AI Max search doubled to 5.28%. Get the channel-by-channel retail benchmarks before Black Friday. SEJ ⋅ AI Search Your Store Is ‘Agent-Ready’ By Default, We Tested 122 Shopify Stores To See What That Actually Means From testing top Shopify stores, Bill Hunt found agents can build a checkout, but can’t answer basic delivery or cancellation questions. Bill Hunt October 14, 2026 ⋅ 17 min read Bill Hunt President at Bisan Digital Bio Follow 426 READS In September 2026, Slobodan Manic argued that the text-only versions of websites now being served to AI “strip out the wrong layer.” A markdown mirror describes the page but drops everything a visitor could press, so it becomes “a brochure … served to the user most eager to act.” He ends with a brilliant three-part list that every ecommerce team should adopt: expose what can be done, make it callable, and report what happened. I had been running an audit of the top 122 Shopify storefronts with a related question: not whether AI can find or cite these stores, but whether an agent can actually resolve a purchase decision on them. That means having enough information, context, rules, and the capability to execute to answer the buyer’s questions, commit to the purchase, and deal with what comes after. His article prompted me to go further. Instead of counting which tools the stores declared, I called them. This article reports what we found, confirms his argument, shows where our results differ, and outlines what both merchants and Shopify can do about it. The platform installed the actions, not the merchants. Manic reports that Shopify switched on WebMCP tools for every Liquid storefront on August 5, and that “the merchants did not build it.” Our data shows the same thing at scale: If you count adoption of agents.md as a sign of merchant readiness, you are really measuring Shopify’s rollout schedule. GEO does not address the action. Manic writes that GEO “has absolutely nothing to do with the action.” Our data backs this up. Most stores publish the facts an agent needs, and Shopify’s checkout works. What breaks is the step in between: turning a published policy into an answer the agent can use when making its decision. The return window appears as prose on a policy page, but the agent can’t retrieve it when asked. Being cited or ranked doesn’t fix that. The semantic floor is broken, including on product pages. He cites WebAIM’s 2026 finding that unlabelled form inputs appear on 51% of home pages, empty links on 46.3%, and empty buttons on 30.6%. We ran the same checks on the product page, where the purchase action actually lives, for the 68 stores we could retrieve: The buy button itself is usually fine. When it is present, it is a native, named <button> 92% of the time. The problem is around it. If two-thirds of size selectors have no label, an agent cannot reliably tell which dropdown sets the size and which sets the color. This is where UX, accessibility, and agent readiness must work together because they depend on the same markup. A labeled size selector, a named buy button, and an announced “added to cart” message serve a screen-reader user, a keyboard user, and a browser agent at once. The conflicts stem from a visual-only design: swatches with no names and confirmations that only appear on screen. Fix the markup once and all three benefit. Huge props to Shopify as nobody else has deployed an agent-callable commerce surface at this scale, and that deserves recognition before any criticism. Across our sample, 89 of 122 stores answered a standard MCP tool listing. Shopify says its catalog now spans “billions of products across millions of merchants,” and that orders from AI-powered searches grew nearly 13 times year over year in Q1 2026. Merchants on the platform received catalog search, cart, checkout, and policy lookup tools without writing a line of code. The quality is also better than I expected. We ran a live, non-completing checkout on every store that exposed the Universal Commerce Protocol (UCP) endpoint: create a checkout for one in-stock item going to a Boston address, read it back, cancel it, then cancel it again. We never completed a purchase or sent payment details. This bears directly on the third item on Manic’s list. He describes an agent that re-submitted a form because the only confirmation was “rendered for eyes.” On Shopify’s tool layer, that failure is handled: the second cancel is refused with a machine-readable error instead of being silently repeated. On that test, Shopify’s tools already report back to the agent properly. These are differences of timing and scope, not disagreements. 1. We Tested A Different Door, And Shopify Has Since Finished His Shopify offers agents two routes into checkout. The first is a server endpoint that an agent calls directly, with no browser (UCP over MCP). The second is WebMCP: tools that register inside the shopper’s own browser tab. Manic tested the browser route on August 6 and found that “the buying path died on an internal error.” At that point, the browser tools covered the storefront and cart, and the checkout step mostly moved the tab to the checkout page. Our updated September test used the server route. Checkout creation succeeded on every store we gave a real, in-stock item, with a median response time of about 1.1 seconds. The only failures came from our own stale data. We had collected item IDs a few weeks earlier. By test day, five of those items had sold out, and two IDs no longer existed. The stores rejected them with clean out_of_stock or invalid codes (“The merchandise with id … does not exist”). So our result does not show that his defect was fixed. We were testing a different route. That small failure is still a lesson for merchants, because agents work the same way we did. An assistant may hold an item ID from a product feed, an earlier conversation, or a shared cart, and try to buy it days later. A sold-out item is normal commerce, as long as every surface an agent reads reports availability promptly. A vanished ID is avoidable. It usually happens when a product is deleted and re-created, or re-imported, instead of edited, and the agent then has to start its search over. Keep variant IDs stable, and keep inventory current across your feeds and catalog. Then, on September 28, Shopify announced support for WebMCP in checkout. Browser agents can now “read and update Shopify checkouts,” covering “the full shopping journey… through checkout and order confirmation,” with no merchant configuration. On September 30, we ran a new test and opened live checkouts at a sample of sites in a browser, with WebMCP exposed to the page. Both registered the four announced tools: get_checkout, update_checkout, complete_checkout, and navigate_to_storefront. A read-only get_checkout call returned the full checkout: line items, totals with tax, fulfillment method, and coded messages showing which fields were still missing. We did not submit an order, but the browser route that failed for Manic now reaches and reads the checkout properly. The tool descriptions also answer his point about agents repeating actions. They tell the agent to “use get_checkout to verify the outcome” and “do not resubmit while status is complete_in_progress or completed.” That’s the “report what happened” requirement, written directly into the tools. Every one of the 74 checkouts came back with the status requires_escalation and a message flagged requires_buyer_input. This matches the rule in Shopify’s default agents.md that checkout requires human approval. That is a reasonable safety choice, and it changes what “agent-ready” means. The new browser tools keep the same rule: complete_checkout is to be used “only after the buyer authorizes the purchase,” and control goes back to the buyer for steps such as 3D Secure. On Shopify today, the agent builds a priced cart, and a person commits the money. An agent can act on a decision only after making one. Manic’s framework covers the doing. Our audit asked whether the agent has what it needs to make a decision and whether it can undo the result. On both, the answer was mostly no, and the September 28 release doesn’t change that. In the browser checkout, the refund terms still reach the agent only as a link to the refund policy page, not as values. get_checkout reads “post-completion order details,” but no tool cancels or changes the order. At one site, the checkout also contained a line item the shopper hadn’t chosen, “Free Returns Coverage” for $2.98, added by the store. The agent can see it, but nothing in the checkout data marks it as optional or explains how to remove it. That’s a rule the buyer needs before committing, and it’s exactly the kind of thing the tools don’t explain. This is the part I most want ecommerce leaders to take away. It is easy to look at a Shopify store today and conclude it is agent-ready: the agents.md file exists, the UCP profile exists, the MCP tools answer, and many readiness checkers will score the store well. What the defaults don’t give you: declared is not the same as done. The agent can’t find the answers to buyers’ questions. Every store with Shopify’s policy tool was asked six questions that a real buyer would ask. We then ran a control test with broad, policy-style questions. Return windows are the one clear success. For everything else, the content is there, and the tool can reach it when the question sounds like a policy page. It cannot reach it when the question sounds like a buyer. One store’s FAQ says “free standard ground shipping on U.S. orders over $75,” and that answer never comes back for a question about what shipping costs. Worse, when asked about cancellation, two stores instead returned their shipping FAQ. A wrong answer that looks confident is more dangerous than no answer, because an agent has no reliable way to tell it doesn’t address the question. Delivery date is missing everywhere. A live checkout priced shipping to Boston for 36 of 74 stores, so for half the sample you can get the cost, but only by starting a purchase. None of the 74 returned an arrival estimate or a carrier, even though the protocol has fields for both. “When will it arrive?” could not be answered through any agent path on any store we tested. Pickup is invisible. All 93 stores with a UCP fulfillment declaration list shipping as the only method; none declares pickup. That includes retailers that clearly offer it. A store that documents Pickup in Store, yet its agent endpoint answers “No pickup locations available.” Another offers Click & Collect and declares shipping only. Here, the agent gets a clear, machine-readable answer that is wrong, and it will send the customer to delivery when same-day collection was available. Most policy information isn’t machine-readable. 70% of the stores whose policy text we could read state a return window in prose. Only 7% of product pages expose it as structured merchantReturnDays.
Source: Search Engine Journal
This article has been carefully curated and reformatted for educational and informational purposes. Full credit goes to the original publisher.
📚 Visit more helpful articles on Joab Peters Blog
No comments