agenticweb.tools Tools Studio

Tool for Humans · API for agents · no LLM

Agent-Readiness Checker: is your site ready for AI agents?

A free agent-readiness checker: enter a website address and get a report on how well AI agents and AI search crawlers can fetch, read and use the site. It reads the page you enter and up to five more the way agents do: no JavaScript, no login. You get three scores, Access, Readability and Actionability, and an overall one with a published formula, the token cost to read each page, what a browser agent sees on it, and the fixes that would raise the score most. Free forever, no account.

For agents: /check/llms.txt is the complete API reference in markdown. Contract: /openapi.yaml (OpenAPI 3.1). The API reference is also below.

44 checks, 3 scores

What it checks

  • Crawl access: final status and redirect chain, HTTPS, robots.txt present and parseable, a verdict for each of 15 AI crawlers and agents for the exact path, the sitemap, noindex
  • Agent discovery: an llms.txt checker (/llms.txt present, served as text, shaped like the convention), /llms-full.txt, an OpenAPI document, /.well-known/api-catalog, a markdown version of the page, MCP hints
  • Content without JavaScript: text in the initial HTML against script weight, an empty app root such as <div id="root"></div>, one <h1>, title and meta description, canonical
  • Honest errors: a made-up address must answer 404 or 410, not 200 with the homepage (a soft 404); the same under /.well-known/
  • Structured data and sharing: valid JSON-LD with its @types, Open Graph title, description and image, a language attribute
  • Delivery: compression, caching headers, response time, HTML size
  • Signal to noise: the token cost to read each page: tokens an agent downloads against tokens of main content (estimated as characters ÷ 4), what the rest is made of (navigation, markup, inline scripts and JSON, inline styles and SVG), how far into the HTML the content starts, and what a Markdown version would save
  • Controls and names: what a browser agent sees, as an approximation of the accessibility tree built from the served HTML: links, buttons and form fields with roles and names, controls with no usable name, the same name on a dozen links, click targets hiding in a <div>, form fields without labels
  • Structure and barriers: landmarks such as <main> and <nav>, heading levels, a skip link; CAPTCHA markers, a sign-in wall, a cookie-consent overlay, content inside a <canvas> or a frame from another site

Page-level checks run on up to six pages: the one you enter and up to five more, sampled from the sitemap (or the page's links) by a fixed rule. The report gives three scores, Access, Readability and Actionability, and an overall one.

The AI crawlers checked in robots.txt: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Google-Extended, Googlebot, Bingbot, CCBot, Applebot-Extended, Bytespider, meta-externalagent.

One call, no key

The same report by API

curl -X POST https://agenticweb.tools/api/site-checks \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com/pricing"}'

{"url":"https://example.com/pricing","score":74,
 "scores":{"access":88,"readability":61,
           "actionability":70,"overall":74},
 "topFixes":[{"id":"content-share","gain":5.8,
   "pages":{"affected":4,"checked":6}, …}, …],
 "checks":[{"id":"canonical","status":"warn",
   "points":0.33,"pages":{"affected":4,"checked":6},
   "detail":"4 of 6 sampled pages have no …",
   "fix":"Add a canonical link …"}, …],
 "pages":[{"path":"/pricing","summary":"An agent
   spends about 38,000 tokens to read 1,200
   tokens of content: 3%.","tokens":{…},
   "actionability":{…}}, …]}

How it behaves

  • What it fetches. The page you enter, up to five more pages of the same site, and the well-known files beside them, such as robots.txt, the sitemap, llms.txt and two made-up addresses that test your 404 page: at most 20 requests in 30 seconds. Every request names itself as AgenticWebToolsChecker/1.0 (+https://agenticweb.tools/check), sends no cookies, and nothing it reads is stored.
  • Three scores, one formula. Access, Readability and Actionability are each 100 × earned ÷ possible weight over their own checks: a pass earns the weight (1 to 3), a warning half, a fail nothing, averaged over the sampled pages; info results are left out. Overall is 40% Access, 35% Readability, 25% Actionability.
  • Not a ranking forecast. The scores say whether agents can fetch, read and use the site. They do not predict how it will rank or whether an AI system will cite it.
  • Several pages, not one. The other pages are chosen by a fixed rule: the home page, then one page from each top-level section, shortest path first. Nothing robots.txt disallows is fetched, and a page that fails to load is reported, not fatal.
  • Token cost to read a page. One sentence per page, such as “An agent spends about 38,000 tokens to read 1,200 tokens of content: 3%”, with the download split into main content, navigation and other text, markup, inline scripts and JSON, inline styles and SVG. Tokens are estimated as characters ÷ 4; no tokenizer and no model is involved.
  • What a browser agent sees. An outline of the accessibility tree as the served HTML gives it, how many links, buttons and form fields have a name an agent can use, and the ones that do not, each with the element and the fix.
  • Deterministic. The same site and the same answers always give the same report. No language model reads the site or writes the findings.
  • Public pages only. The checker refuses private and internal addresses, IP addresses, credentials in the URL and ports other than 80 and 443, and checks every redirect by the same rules.
  • Short quotes only. It reads up to 1.5 MB per file and never returns a file whole: a report quotes at most 200 characters at a time.
  • Shareable. https://agenticweb.tools/check?url=https://example.com opens this page and runs the check. The report can be copied as Markdown.
  • What it cannot see. A firewall that blocks bots by user agent, anything behind a login, and whatever only appears after JavaScript runs. The agent's view is the tree of the HTML as served: controls a script creates later are not in it, and an empty app shell shows an agent nothing to act on.
  • Keeping it out. Refuse the user agent AgenticWebToolsChecker at your server; the report then says the page could not be fetched.

API reference

Base URL https://agenticweb.tools/api. JSON in, JSON out (Content-Type: application/json). No API key, no account, nothing stored. Responses are Cache-Control: no-store. A call takes up to about 30 seconds.

POST /api/site-checks

Run one check. Body {"url": "https://example.com/some/page"}. Returns 200 with the report.

  • url string, required, 1–2048 characters: a public http:// or https:// address on the default port with a domain name as host. Without a scheme it is read as https://. The path and query are kept: the checks are for that exact page.

Errors: 400 invalid_input, 400 invalid_url (scheme, credentials, port, IP address, private or invalid host name), 400 bad_request, 403 cross_site, 422 unresolvable_host, 429 rate_limited, 429 too_many_checks, 503 busy.

A site that is down, slow or blocks the checker is not an API error: the call answers 200 with a report whose page-status check is fail.

The report

  • url, finalUrl (after redirects, or null), checkedAt, durationMs
  • score 0–100 (overall), scores: {overall, access, readability, actionability}, and formula, the formula in words
  • dimensions: [{id, label, score, weight, summary, pass, warn, fail, info}] for access, readability, actionability
  • groups: [{id, dimension, label, score, pass, warn, fail, info}] for crawl, discovery, errors, delivery, content, tokens, structured, controls, structure, barriers
  • topFixes: [{id, label, dimension, status, gain, pages, detail, fix}], the five checks whose fixing would raise the overall score most
  • checks: [{id, group, dimension, label, status, weight, points, scope, pages, detail, fix}]; status is pass, warn, fail or info (reported, not scored); scope is site or page; for a page-level check pages is {affected, checked, examples} and points the share of the weight earned across pages
  • pages: [{url, path, source, status, error, score, summary, findings, tokens, actionability}], the entered page and up to five sampled ones; tokens has rawTokens, mainTokens, parts, contentSharePercent, where the main content starts and the Markdown saving; actionability has the counts, the offenders and the outline as [{depth, role, name}]
  • sampling, and the site figures tokens and actionability
  • bots: [{name, operator, purpose, allowed, group, rule}], the robots.txt verdict for each AI crawler; purpose is search, assistant or training
  • redirects: [{url, status}]; requests: {used, max}; limits; userAgent

Check ids

page-status, https, robots-txt, robots-agents, robots-training, robots-sitemap, sitemap, noindex, llms-txt, llms-full-txt, openapi, api-catalog, markdown, mcp, static-text, app-shell, h1, title-description, canonical, soft-404, well-known-404, json-ld, open-graph, lang, compression, caching, response-time, html-size, content-share, content-position, markdown-saving, control-names, duplicate-names, weak-names, click-targets, form-labels, form-quality, landmarks, heading-order, skip-link, captcha, login-wall, consent-overlay, opaque-content.

Errors

Every error is JSON {"error": "<code>", "message": "<text>"}; branch on error.

StatuserrorMeaning
400invalid_inputNo url in the body, or it is empty or longer than 2048 characters
400invalid_urlNot an address the checker fetches; message names the rule
400bad_requestBody is not valid JSON
403cross_siteA browser on another website made the request
422unresolvable_hostThe host does not resolve to public internet addresses only
429rate_limitedToo many checks; wait Retry-After seconds
429too_many_checksTwo checks from this address are already running
500internal_errorUnexpected server error
503busyThe checker is at its limit of checks running at once; retry after Retry-After seconds

Rate limits

Per client IP: 10 checks per 10 minutes on each server instance, 30 per hour across all instances, and 2 running at the same time. Over a limit: 429 with Retry-After.

Privacy

Nothing about a check is stored: not the URL, not the report, not what was fetched. For abuse prevention each call is logged privately with IP and user agent (not the request body) and deleted after 90 days. No LLM reads, filters or summarises anything.

Questions people ask

What is an agent readiness checker?

A tool that tests whether AI agents and AI search crawlers can discover, read and use a website. This one fetches a page the way they do, without running JavaScript, and reports on crawl access, discovery files such as llms.txt, the text in the HTML, error handling, structured data and delivery.

Is my site ready for AI agents?

Enter its address at https://agenticweb.tools/check. A site is ready when agents are allowed in by robots.txt, the main text is in the HTML the server sends, missing pages answer 404, and there is something for agents to start from: a sitemap, an llms.txt file, structured data, and an OpenAPI document if you have an API.

How do I check my llms.txt file?

Run the check on any page of the site. It fetches /llms.txt and tells you whether it exists, whether it is served as text/plain or text/markdown rather than an HTML page, and whether it has the H1 title, the blockquote summary and the sections of markdown links that the llms.txt convention describes.

Which AI crawlers does it check in robots.txt?

Fifteen: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Google-Extended, Googlebot, Bingbot, CCBot, Applebot-Extended, Bytespider and meta-externalagent. Each gets its own verdict for the path you entered, with the rule that decided it, using the longest-match rules of the Robots Exclusion Protocol (RFC 9309).

Why does my JavaScript app score badly?

Most AI crawlers do not run JavaScript. If the server sends an empty <div id="root"></div> and the text arrives later, they read an empty page. Server rendering, static generation or prerendering puts the text in the first response.

Does it use AI to judge my site?

No. Every check is a deterministic rule, the score is a published formula (a pass earns the check's weight, a warning half, a fail nothing; info results are left out), and nothing calls a language model. It is free and needs no account.

Is there an API for checking agent readiness?

Yes. POST https://agenticweb.tools/api/site-checks with a JSON body such as {"url": "https://example.com"} returns the full report as JSON: the score, a summary per group, every check with its status, detail and fix, and the robots.txt verdict for each AI crawler. No API key is needed. The complete reference is at /check/llms.txt.

Is the Agent-Readiness Checker free?

Yes. It is free, with no account and no sign-up, in the browser and through the API. Nothing is stored: a report exists only in the response.

What are the limits of a check?

A check reads the page you enter and up to five more pages of the same site. It makes at most 20 requests and takes at most about 30 seconds. One client can run 10 checks per 10 minutes and 30 per hour, two at a time. The scores measure whether agents can fetch, read and use the site; they do not predict search ranking or citation.

What is the token cost to read a page?

How many tokens an agent downloads to read a page against how many of them are the main content, for example “about 38,000 tokens to read 1,200 tokens of content: 3%”. The report splits the download into main content, navigation and other text, markup, inline scripts and JSON, and inline styles and SVG, shows how far into the HTML the content starts, and estimates what a Markdown version would save. Tokens are estimated as characters ÷ 4; no tokenizer and no model is involved.

What does a browser agent see on my page?

An approximation of the accessibility tree, which is what most browser agents act on, built from the HTML as served. The report shows an outline of landmarks, headings and controls, how many links, buttons and form fields have a name an agent can use, and the ones that do not: unnamed controls, duplicate names, unlabelled fields, click targets that are not real buttons, and barriers such as CAPTCHAs. Controls that JavaScript creates after load are not in it.

Related tools: Human Response Poll, the poll API for fixed-choice questions and A/B image tests; Ask Humans, the open-question API for free-text answers; Structured Forms, the form and survey API for typed, validated answers; Run Page, a live status page for agent runs; Agent Profiles, a public page that says who an agent is and who oversees it. All AgenticWeb.Tools.