AI CRAWLER CHECK

Can AI assistants read your website?

The check reads your robots.txt and your llms.txt, looks at how your site answers the crawlers of ChatGPT, Claude and Perplexity, and counts the visits that reach you from AI assistants in Google Analytics.

Start free See pricing

✓ Included on every plan, Free too✓ Read-only, nothing on your site changes

ai-crawler-check/busynessprofile.comCrawl of 4 Oct 2026

99of 100

llms.txtDead links

One finding, low impact

llms.txt lists dead URLsFound on busynessprofile.com in the crawl of 4 Oct 2026. Fix: remove or correct the dead addresses.

WHAT IT CHECKS

What decides whether AI assistants can use your site.

Which AI crawlers your robots.txt blocksThe check reads your robots.txt and tells you, crawler by crawler, whether each one may read your pages. It knows 22 AI and search crawlers: 9 that find and cite pages, 9 that collect text for training, and 4 that fetch a page because a person asked.

✓GPTBot, OAI-SearchBot and ChatGPT-User (OpenAI)✓ClaudeBot, Claude-SearchBot and Claude-User (Anthropic)✓PerplexityBot and Perplexity-User (Perplexity)✓Googlebot, Google-Extended and Bingbot✓Applebot, Applebot-Extended and CCBot

Source: Your robots.txt

How your site answers the crawlersWhen you ask for it, the check fetches pages with the crawlers’ published user agents and compares what it gets with what a normal visitor gets, so a firewall that blocks them shows up. Requests carry an X-TouchTargets-Diagnostic header so you can tell them apart.

✓Pages fetched with each crawler’s published user agent✓The response compared with a normal visit✓A refusal from a crawler whose address ranges are published is marked inconclusive, not blocked

Source: Our crawler

WHAT IT FOUND

The finding on our own site.

From the crawl of busynessprofile.com on 4 Oct 2026.

#ISSUEIMPACTPAGES

WHY IT MATTERSAI assistants that read llms.txt are sent to pages that return errors.

THE FIXRemove or correct the dead addresses in llms.txt.

SOURCEOur crawl, 4 Oct 2026Step-by-step fix

PRICE

Included on every plan.

EVERY PLANFreeThe AI crawler and referral check is included on every plan, including Free.

✓ Which AI crawlers your robots.txt blocks✓ Your llms.txt and its dead links✓ AI visits from Google Analytics 4, when it is connected

Start free

PROOF

Numbers from our own directory.

21,898pages crawled in the first audit (30 Sep 2026)Source: first full crawl of busynessprofile.com on production, 30 Sep 2026

45site health score, up from 36 in the crawl beforeSource: TouchTargets site audit of busynessprofile.com, crawls of 3 and 4 Oct 2026

1,040search clicks in August, up from 119Source: Google Search Console, busynessprofile.com

3×enquiries over-counted in GA4Source: Google Analytics 4 events, busynessprofile.com

WORKS WITH

See the rest of the site, too.

See plans and prices →

AUDITSite auditCrawls your site and ranks every fix by how much fixing it helps.→TRACKSearch Console reportShows why your search clicks moved.→

Questions about the check

Yes. It is included on every plan, including Free.

No. It only reads. Nothing on your site changes unless you approve it.

22 AI and search crawlers from OpenAI, Anthropic, Perplexity, Google, Microsoft, Apple and others, with what each one is for. The list needs upkeep because crawler names change.

START FREE TODAY

Find out whether AI can read your site.

Start free See pricing

Talk to Rakesh directlyNo sales team. You speak with the person who builds it.