{"uid":"cap_prxGNmxZJDADU6aE38qBl","slug":"llms-txt-auditor-31492ea0","name":"LLMs.txt Auditor","description":"Pay-per-call data & tool APIs","url":"https://api.timzinin.com/api/llms-txt-auditor","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"items":{"type":"array","maxItems":100},"maxConcurrency":{"type":"integer","maximum":50,"minimum":1}}},"responseSchema":{"type":"json","example":{"meta":{"run_ms":1200,"settled":true,"item_count":1},"results":[{"found":true,"input":"nytimes.com","domain":"https://nytimes.com","summary":"no llms.txt, blocks 8/9: GPTBot, ChatGPT-User, ClaudeBot, anthropic-ai, Google-Extended, PerplexityBot, CCBot, Bytespider.","scrapedAt":"2026-07-26T13:20:22.136Z","aiCrawlers":{"CCBot":"blocked","GPTBot":"blocked","Amazonbot":"allowed","ClaudeBot":"blocked","Bytespider":"blocked","ChatGPT-User":"blocked","anthropic-ai":"blocked","PerplexityBot":"blocked","Google-Extended":"blocked"},"hasLlmsTxt":false,"llmsTxtBytes":0,"hasLlmsFullTxt":false}]}},"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.01","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.01/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.01","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.01","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_cHQ6o9yhW-Ig4LJSjKeW-","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.01","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Checks a domain for the presence of llms.txt and robots.txt AI crawler directives, reporting which AI bots are allowed or blocked","exampleAgentPrompt":"Can you check whether openai.com has an llms.txt file and which AI crawlers — like GPTBot, ClaudeBot, and PerplexityBot — it allows or blocks?","exampleUseCases":[{"title":"Pre-scrape AI crawler policy check","prompt":"Before I try to crawl techcrunch.com with my AI agent, can you check whether it has an llms.txt and whether it blocks GPTBot or ClaudeBot?"},{"title":"Competitor content access audit","prompt":"I want to know if our competitor's site salesforce.com blocks AI crawlers like Amazonbot, PerplexityBot, and Google-Extended — can you run an llms.txt audit on it?"},{"title":"Research dataset sourcing validation","prompt":"I'm building a training dataset and need to know if wikipedia.org allows CCBot and Bytespider — check its llms.txt and robots.txt AI crawler rules for me."}],"resultDescription":"A JSON object containing: whether llms.txt and llms-full.txt exist and their byte sizes, a per-crawler verdict (allowed/blocked) for major AI bots including GPTBot, ClaudeBot, ChatGPT-User, Amazonbot, Bytespider, PerplexityBot, Google-Extended, CCBot, and anthropic-ai, a human-readable summary, the canonical domain URL, and the timestamp of when the data was scraped. Also includes run time in milliseconds and payment settlement confirmation.","failureModes":["Domain not found or unreachable — found:false with empty crawler data","Network timeout fetching robots.txt or llms.txt — may return partial or stale results","Invalid domain format in input — likely returns an error or empty result","Domain exists but has no robots.txt and no llms.txt — all crawlers reported as allowed by default","Rate limiting or firewall on target domain may cause scrape failure"],"whenToPreferThis":"Use this endpoint when you need to quickly determine AI crawler access policy for a specific domain without manually fetching and parsing robots.txt or llms.txt yourself. Ideal for agents that need to validate whether a site permits AI scraping before attempting it, for dataset curation pipelines that must respect publisher permissions, or for monitoring whether a domain's AI crawler policy has changed. At $0.01 per call it is cost-effective for per-domain spot checks.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-14T18:54:57.395Z","isFirstParty":false}