{"uid":"cap_DVpc58vxRZXjhD92E71ly","slug":"crawlwitness-ai-consent-checker-71bf1d35","name":"CrawlWitness AI Consent Checker","description":"What a website allows for AI (robots.txt per AI bot, RSL, IETF AIPREF, TDMRep, noai, llms.txt, C2PA), with signed EAS receipts anchored on Base and an attested history of every signal. Paid calls use x402 (USDC on Base or Solana): no account, no API key.","url":"https://api.crawlwitness.com/v1/senales?utm_source=zero.xyz","method":"GET","headers":{},"bodySchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["input"],"properties":{"input":{"type":"object","required":["type","method"],"properties":{"type":{"type":"string","const":"http"},"method":{"enum":["GET"],"type":"string"},"queryParams":{"type":"object","required":["dominio"],"properties":{"pagar":{"enum":["true","false"],"type":"string","description":"true pays directly instead of using the daily free quota"},"fresco":{"enum":["true","false"],"type":"string","description":"true forces a fresh analysis instead of the cached one"},"dominio":{"type":"string","maxLength":253,"minLength":3,"description":"Domain to check, without scheme or path (e.g. nytimes.com)"}}}},"additionalProperties":false},"output":{"type":"object","required":["type"],"properties":{"type":{"type":"string"},"example":{"type":"object"}}}}},"responseSchema":{"type":"json","example":{"pago":{"red":"eip155:8453","atestacion":{"uid":"0x…","url_easscan":"https://base.easscan.org/…"},"transaccion":"0x…"},"origen":"cache","dominio":"nytimes.com","informe":{"tdm":{"reserva":false},"robots":{"encontrado":true,"estado_http":200,"licencias_rsl":[]},"resumen":{"motivos":["robots.txt bloquea GPTBot, ClaudeBot, CCBot y Google-Extended"],"bots_total":24,"exige_pago":false,"puntuacion":55,"entrenamiento":"reservado","bots_bloqueados":18,"recomendaciones":[]},"llms_txt":false},"analizado_en":"2026-10-30T12:00:00Z"}},"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.005","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.005/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.005","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.005","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_cw0biUiN-bfAgXKKRrAC3","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.005","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Checks a domain's AI crawling consent and licensing signals, returning a 0-100 compliance score, training stance, and a signed on-chain receipt proving what was checked and when.","exampleAgentPrompt":"Before you start ingesting content from bbc.com into our RAG pipeline, can you check CrawlWitness to see if the site allows AI crawling — I want a fresh check, not a cached result — and get back the consent score and a signed proof of what was verified?","exampleUseCases":[{"title":"RAG pipeline pre-ingestion consent gate","prompt":"I'm about to pull articles from theguardian.com into our knowledge base — can you run a CrawlWitness check on theguardian.com and tell me their AI training stance and consent score before we proceed?"},{"title":"LLM training dataset compliance audit","prompt":"We're building a training dataset and want to use content from arxiv.org — check their AI consent signals including robots.txt, llms.txt, and TDMRep, and give me a fresh score with the signed on-chain receipt so we have proof we did our due diligence."},{"title":"Automated crawler consent enforcement","prompt":"Set up a check so that whenever our scraper is about to hit a new domain, it first verifies on CrawlWitness whether that domain permits AI crawling — flag anything scoring below 50 for human review."}],"resultDescription":"Returns a 0-100 AI consent score, a training stance label (e.g. allowed, restricted, prohibited), granular signal breakdown per standard (robots.txt per AI bot, RSL, IETF AIPREF, TDMRep, noai meta tags, llms.txt, C2PA), and a cryptographically signed EAS attestation receipt anchored on the Base blockchain as tamper-proof evidence of what was checked and when.","failureModes":["400 error returned when domain is invalid or malformed (not charged)","502 error returned when the target site is unreachable during analysis (not charged)","Cached result returned if fresco=false and a recent check exists — may be stale for rapidly changing policies","Daily free quota exhausted — use pagar=true to bypass with direct USDC payment"],"whenToPreferThis":"Choose CrawlWitness when your agent needs to verify AI crawling consent before ingesting, training on, or scraping a website, especially when you need verifiable, auditable proof of compliance anchored on-chain. It is the right choice when legal and ethical AI data sourcing is required, when building RAG pipelines with third-party content, or when operating in regulated environments where a signed receipt of consent verification is needed. Prefer this over manual robots.txt parsing because it checks multiple AI-specific standards simultaneously (AIPREF, TDMRep, RSL, llms.txt, C2PA) and provides a normalized score.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-11-01T06:30:24.643Z","isFirstParty":false,"canonicalSlug":"crawlwitness-ai-consent-checker-71bf1d35"}