Field notes
A high static score does not mean agents can buy: static checks and real walks measure different things
Two instruments, one store. The static checker said "UCP profile exists: pass." The walk record said "stopped at signup, 20/100", which we later found was most likely a false positive. The two instruments measure different things in the first place.
Correction (2026-09-15): This article previously relied on one of our runs against Shopify's public demo store (hydrogen.shop), recorded as "stopped at a signup CAPTCHA, 20/100". We no longer use that record as evidence. The run saved no screenshots or HTML, and the same detection does not reproduce on hydrogen.shop today. On review, our detector at the time flagged a CAPTCHA whenever the page HTML merely contained strings such as "recaptcha", and this run's record consists only of such string matches. We cannot say a CAPTCHA was actually shown, and we now consider it most likely a false positive. Our 2026-09-02 note kept the number "for reference"; that was not enough. The affected sentences have been rewritten.
The setup
We never walk a store without the owner's consent, so the subject here is Shopify's own public demo store, hydrogen.shop. The walk side is our published demo report from 2026-07-30. On record: score 20/100, 13 actions, ~$0.54, stopped at signup because our detector flagged a CAPTCHA. Correction (2026-09-15): that flag was based only on strings such as recaptcha in the page HTML; no screens were saved and it does not reproduce today, so it was most likely a false positive. Nothing after signup was walked, so whether cart and checkout pass is unknown.
The static side: on 2026-08-20 we ran the same store through Cloudflare's Is Your Site Agent-Ready? scanner. Verdict: Level 1, "Basic Web Presence." robots.txt and sitemap pass, most agent standards fail — and notably, the UCP check passes, because hydrogen.shop really does serve a /.well-known/ucp declaration.
Side by side
| Question | Static checker | Real walk |
|---|---|---|
| Is a UCP declaration served? | Pass | Not measured |
| robots.txt / sitemap in order? | Pass | Not measured |
| Can the signup form be passed? | Not measured | Stopped at CAPTCHA on record (most likely a false positive) |
| Cart and checkout reachable? | Not measured | Not reached (run was stopped, so unknown) |
Declaring a protocol and being traversable are separate facts; neither implies the other. We originally wrote that this very store declared UCP yet the agent could not get through signup; that claim is withdrawn, because the stop behind it was most likely a false positive. What the episode does show is that a walk without saved screens cannot be checked afterwards. The gap between declaring and being buyable also shows up at industry scale: UCP Checker's July 2026 report counted 11,414 verified stores and zero with a confirmed payment implementation (source).
Static checks are still worth doing
They are the cheap first step — the questionnaire and blood test before the treadmill. Fix discoverability with a free one-minute scan, then find out whether the path actually passes by walking it. For a Shopify store, the Shopify playbook separates the rows a merchant can fix from the platform residual their admin panel cannot reach. Our opt-in study walks consenting stores only and returns a free report with the stop reason — no CAPTCHA solving, no completed purchases. See the Japan EC agent-readiness study.
Try it on your own site
One free diagnosis per site, for consented sites only. We never walk a store without permission, never click purchase confirmation and never solve CAPTCHAs.