{"uid":"cap_uCg6Fg_Cj8LtP9mnO0vCW","slug":"ai-crawler-access-check-ddc326b2","name":"AI Crawler Access Check","description":"AI Crawler Access Check: Who Gets a 403 — Reads the results of probing a website with AI-crawler user agents (status codes per bot, a control request, sometimes robots.txt or the site's settings) and says what they really show as JSON - whether training bots, search bots and user-triggered agents are open, blocked, mixed or unknown, what is behind a refusal, and whether the robots.txt that answered is the site's own file. Knows the traps - a bare bot name proves nothing and lies differently for…","url":"https://api.aiskills402.com/v1/skills/crawler-access-probe/file?utm_source=zero.xyz","method":"GET","headers":{},"bodySchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["input"],"properties":{"input":{"type":"object","required":["type","method"],"properties":{"type":{"type":"string","const":"http"},"method":{"enum":["GET","HEAD"],"type":"string"},"headers":{"type":"object","additionalProperties":{"type":"string"}}},"additionalProperties":false},"output":{"type":"object","required":["type"],"properties":{"type":{"type":"string"},"example":{"type":"object"}}}}},"responseSchema":{"type":"json","example":{"ok":true,"file":{"name":"SKILL.md","content":"<the SKILL.md file>","encoding":"utf-8"},"name":"AI Crawler Access Check: Who Gets a 403","skill":"crawler-access-probe","sha256":"<sha256 of SKILL.md>","receipt":{"tx":"0x<settlement transaction>","settled":true},"version":"1.0.0","redownload":{"url":"<api>/v1/purchases/<token>","token":"<64 hex chars>"}}},"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.05","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.05/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.05","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.05","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_l6a0dDNGEQO1_Bzu2PwuX","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.05","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Returns a SKILL.md file describing how to interpret AI-crawler probe results — classifying which bot categories are blocked, allowed, or mixed on a given website.","exampleAgentPrompt":"I want to understand which categories of AI crawlers — training bots, search bots, and user-triggered agents — are actually allowed or blocked on a site; can you get me the skill guide that explains how to read those probe results and what the status codes really mean?","exampleUseCases":[{"title":"Audit site openness to AI training bots","prompt":"I ran a crawler probe on a site and got mixed status codes back — can you pull the skill file that explains what those results mean and whether training bots like GPTBot or CCBot are really being blocked or just selectively allowed?"},{"title":"Verify robots.txt authenticity for a domain","prompt":"I need to know if the robots.txt that answered my crawler probe is actually the site's own file or something else — get me the guide that explains how to detect that and what it means for bot access."},{"title":"Understand refusal reasons for AI user agents","prompt":"A site is returning 403s for some AI crawlers but not others, and I can't tell what's behind the refusal — can you fetch the skill document that breaks down what different block patterns actually indicate for training bots versus user-triggered agents?"}],"resultDescription":"Returns a SKILL.md markdown file (with UTF-8 encoding and SHA256 hash) that explains how to interpret AI-crawler probe results — classifying bot categories as open, blocked, mixed, or unknown, explaining what drives refusals, and flagging whether the responding robots.txt is authentic. Also includes a purchase receipt with a blockchain settlement transaction and a redownload URL with token.","failureModes":["Payment not settled — 402 response if USDC payment on Base is not confirmed","Invalid method — only GET or HEAD accepted; other methods return error","File unavailable — if SKILL.md cannot be retrieved, ok field will be false","Network timeout — slow response if upstream settlement is delayed","Invalid input schema — missing required 'type' or 'method' fields returns validation error"],"whenToPreferThis":"Choose this endpoint when an agent needs authoritative, interpretive guidance on AI-crawler HTTP probe results — specifically to classify bot category access (training, search, user-triggered) and understand the semantics behind 403 responses and robots.txt authenticity. Prefer this over raw HTTP probing tools when you need curated, expert-authored interpretation logic packaged as a reusable skill file rather than raw status codes.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-18T06:57:48.638Z","isFirstParty":false,"canonicalSlug":"ai-crawler-access-check-ddc326b2"}