ShelfglanceBot

ShelfglanceBot reads public pages of a store to work out what an AI shopping assistant would find there. It requests at most a dozen URLs per scan and keeps no copy of page content, only the findings.

When a visitor asks for a scan

Most scans happen because someone typed a domain into the box on this site. That request is theirs, so the scan fetches the pages they pointed it at without checking robots.txt first, the same way your browser does when you open a link.

When we re-scan on a schedule

Two things run on a schedule. A store owner can ask to hear from us if their score changes, and those stores are re-scanned about once a month. Separately, every store that has a kept report is re-read on a rolling schedule, oldest first, so the published figures describe stores as they are now rather than as they were when we first saw them.

Nobody is standing there asking for either of those, so both are bound by your robots.txt: before a scheduled re-scan we fetch /robots.txt and stop there if ShelfglanceBot is disallowed.

Turning it off

Add this to your robots.txt and no scheduled re-scan will fetch your pages:

User-agent: ShelfglanceBot
Disallow: /

That also takes an existing report down, and it does not wait for the report to lapse: the next scheduled pass over your store reads the disallow, deletes the stored report without scanning, and does not renew it, so the store leaves the directory and does not come back. A report that nothing renews is deleted three months after the scan that produced it in any case.

What it reads

Your homepage, one product page, robots.txt, sitemap.xml and llms.txt. It does not log in, does not submit forms, and never touches customer names, emails, phone numbers, addresses or orders.

What this site counts when you visit

One line per page view: which kind of page, which site sent you, and a campaign tag if the link carried one. No name, no address, no identifier, nothing that can be joined up into a person or a visit history. It exists because 37,000 requests a day to this site are crawlers and we had no way to tell how many readers were people.

If you would rather not be counted, open this page with ?nopx=1 once and this browser stops sending it. That flag is the only thing we store on your machine, and clearing your site data removes it.