{"uid":"cap_GBMGzO3Qy-lC17GPXyPdc","slug":"halowerk-ai-agent-access-checker-ad6c9802","name":"Halowerk AI Agent Access Checker","description":"Wertet robots.txt fuer 23 benannte KI-Agenten aus, mit korrekter Gruppenauswahl und der Regel, dass die laengste passende Anweisung gewinnt. Prueft zusaetzlich X-Robots-Tag, Meta-Angaben, noai, Bezahlschranken-Hinweise, Lizenzverweise, Nutzungsbedingungen und den TDM-Vorbehalt unter /.well-known/tdmrep.json.","url":"https://tools.halowerk.com/v1/agent/access","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"path":{"type":"string","default":"/","description":"Zu pruefender Pfad, zum Beispiel /blog/artikel-1"},"domain":{"type":"string","description":"Domain oder URL"}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.004","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.004/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.004","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.004","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_MhPs4RTbbw8b9MatFkTbw","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.004","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Checks whether a given URL permits AI agent access by evaluating robots.txt for 23 named AI agents, X-Robots-Tag headers, meta tags, noai directives, paywalls, license references, terms of use, and TDM reservations via /.well-known/tdmrep.json.","exampleAgentPrompt":"Before scraping that article, check whether the site at https://example.com actually allows AI agent access — look at their robots.txt for GPTBot, any noai meta tags, X-Robots headers, and whether they have a TDM reservation at /.well-known/tdmrep.json.","exampleUseCases":[{"title":"Pre-scrape AI crawler compliance check","prompt":"Before my pipeline collects training data from https://news-site.com, check whether AI crawlers are permitted there — look at their robots.txt for known AI agents, any noai or X-Robots-Tag directives, and whether they've filed a TDM reservation."},{"title":"Content licensing audit for AI startup","prompt":"We're building a dataset from 50 sites — can you check https://blog.example.com to see if it allows AI data mining, has any license restrictions, a paywall, or has blocked AI agents like GPTBot or CCBot in its robots.txt?"},{"title":"Agent access policy before agentic browsing","prompt":"My AI assistant is about to read and summarize pages from https://research-journal.org — first check whether that site permits AI agent access, including their TDM reservation, any noai meta tags, and whether they block agents like Anthropic-AI or PerplexityBot in robots.txt."}],"resultDescription":"Returns a structured report covering: which of 23 named AI agents are permitted or blocked by robots.txt (using correct group selection and longest-match-wins rule), X-Robots-Tag header values, meta noai/noindex flags, paywall detection signals, license references found on the page, links to terms of use, and TDM reservation status from /.well-known/tdmrep.json.","failureModes":["URL unreachable or returns non-2xx status — endpoint reports fetch error","robots.txt absent — treated as fully permissive per standard","TDM rep file missing — reported as no TDM reservation","Malformed robots.txt — best-effort parsing with warnings","Timeout on slow sites — may return partial results","Paywalled robots.txt or redirect loops — fetch failure reported"],"whenToPreferThis":"Use this endpoint when an AI agent needs to verify whether it is permitted to crawl, scrape, or train on content from a specific URL before doing so. It is specifically designed for AI agent compliance, covering 23 named AI crawlers with correct robots.txt group selection and longest-match-wins semantics — far more thorough than a generic robots.txt parser. Prefer it when you need TDM reservation checks, noai meta tag detection, and paywall signals bundled in a single call.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-14T18:33:13.614Z","isFirstParty":false}