Does an AI crawler visiting your Shopify store mean it will recommend you?
You may see GPTBot, Googlebot, or PerplexityBot in your store logs and take it as a good sign.
It can be a good sign. But it is not a recommendation.
A crawler visit means a system fetched a page or file from your store. It does not mean ChatGPT, Perplexity, Claude, or Gemini will name your store when a shopper asks what to buy.
Those are different checks. You need both.
A crawler visit shows access, not a recommendation
AI companies use crawlers to fetch public pages. A crawler might read:
- A product page
- A collection page
- Your shipping or returns policy
- Product markup in the page HTML
- A file such as `/llms.txt`
That tells you the crawler could reach that part of the store.
It does not tell you what it understood. It does not tell you whether it stored the page for later use. It also does not tell you whether an assistant will cite or recommend the page in an answer.
OpenAI makes this distinction in its own bot documentation. `GPTBot` crawls content that may be used to improve or train foundation models. `OAI-SearchBot` is used to surface sites in ChatGPT Search. `ChatGPT-User` is different again: it may fetch a page because a user asked for it, but it is not an automatic crawler and does not decide whether content appears in ChatGPT Search.
So a log entry needs context.
If `GPTBot` visits a product page, that is not the same thing as appearing in a ChatGPT Search answer. If `ChatGPT-User` fetches a page, that may reflect one user’s request, not general discoverability.
Think of a crawler visit as someone entering your shop.
They got through the door. You still do not know what they read, whether they found the right product, or whether they would tell someone else to shop there.
A recommendation answers a real shopper question
Recommendations need a direct test.
Say your store sells hiking backpacks. A shopper might ask:
- “What’s a good hiking backpack for a two-day trip?”
- “Which backpack fits a carry-on and holds camping gear?”
- “What lightweight hiking packs have a hip belt?”
- “Does this brand make a backpack for weekend hikes?”
Each question tests something different.
An assistant may know your brand but not connect it to a product need. It may name your store for a brand question but leave you out of a category question. It may recommend a competitor because that competitor’s product page explains fit, materials, capacity, or delivery more clearly.
That is why AI visibility should be treated as counts tied to questions, not as one vague claim.
Useful results look like this:
- ChatGPT named your store in 3 of 9 answers.
- Perplexity named a competitor in 6 answers.
- Claude named your brand when asked directly, but not for category questions.
- Gemini cited your returns policy but did not name a product.
Each result gives you something to inspect.
Descriva tests shopper questions across ChatGPT, Perplexity, Claude, and Gemini. It shows which assistant named your store, which left it out, which brands appeared instead, and which sources the answer cited.
It reports counts. AI readiness is Descriva’s only score.
Check crawler access before testing mentions
First, check whether public pages are open to the crawlers you want to allow.
Open this in a browser:
```text yourstore.com/robots.txt ```
Look for rules that block crawlers from the whole site or from product pages.
For example:
```text User-agent: OAI-SearchBot Disallow: / ```
OpenAI says blocking `OAI-SearchBot` prevents a site from appearing in ChatGPT Search answers, though the site may still appear as a navigational link.
You may choose to block a crawler. That is your call. But if you want open-web discovery through that crawler, the rule creates a clear conflict.
Then open an important product page in a private browser window. Check that it loads without:
- A password page
- A required customer login
- A location gate that hides the product
- A draft or unlisted product
- No product image
- A price of 0
- The product missing from the Online Store sales channel
- Search engine visibility turned off
These details matter for Shopify Catalog too.
Shopify Catalog is separate from ordinary web crawling. For eligible products, it distributes structured product data to Shopify’s agentic channels, including titles, descriptions, options, images, prices, availability, and other attributes. Shopify calls Catalog the authoritative product-data feed for its agentic channels.
This means robots.txt is not a complete answer to AI discovery.
Robots.txt rules affect open-web crawling. They do not replace the product data Shopify agents receive through Catalog.
Check what agents receive from Shopify Catalog
Descriva’s Agent view shows what Shopify Catalog provides for each product. It also flags requirements the product misses.
That includes products that are:
- Draft or unlisted
- Missing an image
- Priced at 0
- Not published to the Online Store
- Hidden from search engines
- Behind a password page
If a product has no category, Descriva can suggest one. It only applies the category after you approve it.
This is worth checking because a product can have a public page and still be poorly prepared for agentic shopping channels.
Take one minute now: open a product you want agents to find, then check its status, image, price, Online Store availability, and search engine listing in Shopify.
Not every crawler claim is genuine
A user-agent string can be copied.
A request may claim to be GPTBot, ClaudeBot, or another known crawler. That alone does not prove the request came from the company behind it.
Descriva checks crawler visits against the IP address ranges each company publishes. It labels visits as:
- Verified — the address matches the company’s published range.
- Unverified — the request claimed to be that crawler, but the address could not be confirmed.
- Self-reported — the crawler identifies itself, but no published address range is available to verify it.
Crawler tracking starts when you switch on Descriva’s theme app embed.
Use this data to answer a narrow question:
Are recognised crawlers reaching the public parts of my store?
Do not use it as proof that an assistant will recommend your products.
AI files help explain the store, but they are not the product feed
Shopify’s current agent-discovery files are `/agents.md`, `/llms.txt`, and `/llms-full.txt`.
Shopify says `/agents.md` is the canonical source. The two `llms` files support older crawlers. These files can provide store context, policies, sitemap information, and discovery endpoints.
They do not replace Shopify Catalog.
Shopify already serves a default `/llms.txt`, so that basic file counts as done. Descriva can also provide a theme template that lists your products, then checks the live file after you paste it into the theme.
We chose that approach deliberately. Descriva does not ask for theme-write permission. That means less access for you to trust. The trade is that theme changes arrive as a template for you to paste, and Descriva reads the live URL afterwards instead of assuming the change worked.
Descriva also serves catalogue files and markdown product and collection pages on your own domain. They update within 10 minutes of a product change.
If crawlers visit but assistants leave you out
Start with the answer, not a guess.
Look at:
- The shopper question
- Which assistant left you out
- Which brands it named instead
- Sources it cited
- Whether your product page answers the same practical need
If the question is about fit, compatibility, materials, delivery, or returns, can a shopper find the answer in clear text on your product or policy pages?
For example, “Trail Pack 30L” does not explain whether a bag fits a carry-on, holds a laptop, works for overnight hiking, or suits a shorter torso. If those details matter to the shopper, they need to be present in the store’s content.
Descriva’s Optimize tab brings the product description, title, search listing, image alt text, FAQ, and specs pack into one place. You can write changes yourself or draft them with AI. Nothing goes live until you press Publish. History keeps the previous version of every published change.
Start with one product
You do not need to test your entire catalogue first.
Pick a bestseller. Choose three questions a shopper would ask before buying it. Then check whether the four assistants name your store and who they name instead.
Every store gets one AI visibility check when it installs Descriva, before choosing a plan. You pick one product. Descriva asks three shopper questions across ChatGPT, Perplexity, Claude, and Gemini.
A crawler visit is useful evidence. It tells you something could reach your store.
The next useful question is simpler: when a shopper asks for what you sell, does the assistant name you?