AI Visibility Hub
Reads robots.txt, llms.txt and structured data for any public website.
Free AI Crawler and llms.txt Checker
Paste a website address below and this reads its robots.txt, its llms.txt and the structured data on its home page, at no cost. It reports which of the eight crawlers AI assistants use are blocked, whether llms.txt exists, and which schema.org types the page carries. No email, no account, and nothing about the site you check is stored.
Run the check
Paste any public website below. This reads its robots.txt, its llms.txt and the structured data on its home page, live, right now.
Reading robots.txt, llms.txt and the home page
We could not read that site.
Check the address and try again.
llms.txt
Structured data (schema.org)
What these eight crawlers are, and why a block matters
A few lines in one file can shut an AI assistant out of a site entirely.
robots.txt is a plain text file at example.com/robots.txt that tells crawlers what they can and cannot fetch from a site. The crawlers this check reads for are:
- GPTBot, OpenAI's crawler, reads sites for ChatGPT.
- ChatGPT-User, OpenAI, fetches a page on the spot when someone asks ChatGPT to open a link.
- OAI-SearchBot, OpenAI, reads sites for ChatGPT's search results.
- PerplexityBot, Perplexity's crawler, reads and indexes the sites its answers cite.
- Google-Extended, controls whether Google's Gemini models can use a site's content, separate from regular Search indexing.
- ClaudeBot, Anthropic's crawler, reads sites for Claude.
- Bingbot, Microsoft's crawler, reads sites for Bing and for Copilot.
- Googlebot, Google's main crawler, reads sites for Search and for AI Overviews.
If robots.txt disallows any of these, that assistant cannot read the site's pages, no matter how good the writing or the schema on them is. This usually happens by accident: a website builder's default file, a security plugin set too aggressively, or a rule copied from another site. The owner has no way of knowing until someone checks.
The check above reads a site's robots.txt directly and reports plainly which of these eight, if any, are blocked.
What llms.txt is
One plain text file that states what a business does.
llms.txt sits at example.com/llms.txt: a plain text summary of what a business does, the areas it serves, its hours, and links to the pages that answer a buyer's specific questions. Some assistants and crawlers read it when it exists. No one, us included, can say for certain how much weight it carries with any one assistant. It is a low cost thing to add, not a replacement for open crawler access or working schema.
A worked example, built for a local trade, is at /ai-seo/llms-txt-for-local-business/. The check above reports whether a site has one.
What schema.org structured data tells a machine
The facts on a page, in a format a machine can parse directly.
Schema, usually written as JSON-LD, is a block of structured data added to a page's code, separate from the text a visitor sees. Common types on a local business site are Organization, LocalBusiness and Service: the business name, address, phone, hours and services offered, stated as fields instead of sentences a machine has to guess at.
The check above reads the JSON-LD on a site's home page and lists every schema.org type it finds. No types found usually means a machine reading that page has only the paragraphs written for a person to work from, and has to guess the rest.
More on getting these three things right is in our robots.txt, schema and llms.txt guide.
Questions owners ask
Short answers, no promises.
Does this check cost anything?
No. It is free, with no account and no email required.
Which crawlers does it check?
GPTBot, ChatGPT-User and OAI-SearchBot from OpenAI, PerplexityBot from Perplexity, Google-Extended from Google, ClaudeBot from Anthropic, and Bingbot and Googlebot from Microsoft and Google.
If llms.txt is missing, does that alone keep an assistant from naming my business?
No single fix guarantees that. A blocked crawler stops an assistant from reading the site at all. A missing llms.txt or missing schema just removes one more thing that could have helped.
Do I need a developer to fix a blocked crawler or missing schema?
Someone who can edit the site's files needs to make the change. Our $397 pilot makes the changes directly, not just a list of instructions for someone else to follow.
Does this tool tell me if ChatGPT or Gemini actually names my business?
No. This check only reads robots.txt, llms.txt and schema. The free six-question check at /ai-check/ asks real buyer questions on ChatGPT, Gemini and Perplexity and reports whether the business was named.
Who runs Conecta Media?
One person, Jorge, a software engineer, based in Chicago. There is no team and no office visit; the work happens on the site itself.
This checks access. It does not check what an assistant actually says.
Two different free checks, for two different questions.
This page checks whether robots.txt blocks the crawlers AI assistants use, whether llms.txt exists, and what schema.org data the home page carries. It does not ask ChatGPT, Gemini or Perplexity whether they actually name the business for a real buyer question.
Our free six-question check does that: it writes six questions a buyer in the business's trade and town would ask, asks all six on ChatGPT, Gemini and Perplexity with live web search, and reports who got named. No account, no cost.
Run the free six-question check
If either check turns up a blocked crawler, missing schema or no llms.txt file, the pilot fixes it, for $397 once, over 14 days: opening robots.txt to the crawlers those assistants use, adding or correcting schema, writing an llms.txt file, and a written page for any buyer question the site missed. See pricing for the full list.