Tools / AI crawlers & agents
See your page the way an AI crawler does
GPTBot, ClaudeBot and PerplexityBot don’t run JavaScript. See the exact text they get, and whether your page is an empty shell.
Free, no signup.
Example result: smoothrecorder.com · checked 2026-09-30
What an AI crawler receives
| Verdict | ✓ Readable without JavaScript |
| HTTP | 200 · 288ms |
| Words in raw HTML | 1993 |
| HTML size | 420 KB |
| Script tags | 17 |
| Title | Smooth Recorder: Record Beautiful Demos on Mac, Without Editing |
Why it matters
AI crawlers read raw HTML
A Vercel study of 500M+ crawler fetches found OpenAI, Anthropic and Perplexity bots don’t execute JavaScript.
Client-rendered pages look blank
If your content appears only after React or Vue runs, AI engines see a loading spinner and move on.
It’s invisible in a browser
Your page looks perfect to you. Nothing warns you that bots got nothing.
What it checks
Words in raw HTML
How much readable text arrives before any JavaScript runs.
Empty-shell detection
Spots the classic <div id="root"></div> single-page-app pattern.
HTML size
Very heavy HTML is slow for every crawler to parse.
Script count
How much of the page depends on JavaScript.
Title
Whether the title is in the HTML or injected later.
Readable text preview
The first 1,500 characters exactly as a crawler reads them.
How it works
Enter a URL
A domain or page address. No install, nothing to add to your site.
We fetch it live
Our servers request the page read-only, the way a crawler would.
Read and fix
Get the results in seconds, with what each one means.
Questions
More in AI crawlers & agents
Check everything at once
The site audit checks your whole site and scores SEO, AI Search and Agent Readiness in about 30 seconds.
Audit my site