{"uid":"cap_XfKJ1H4hI-2sQdhg4SU-o","slug":"agent-miner-tools-web-resource-preflight-check-5c0a5359","name":"Agent Miner Tools – Web Resource Preflight Check","description":"Use before spending tokens, money or crawler capacity fetching an unfamiliar web resource. Resolves redirects (manually, bounded, re-validated per hop) and returns deterministic HTTP/resource facts — final URL, status, content type and length, response time, a bounded HTML title/canonical peek, and a lightweight generic-bot robots.txt indication — so an agent can decide whether a normal web fetch is worthwhile. It does not scrape, render JavaScript, or judge safety.","url":"https://agent-api-production.tim1712.workers.dev/v1/web/resource-preflight","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"url":{"type":"string","maxLength":2048,"minLength":1}}},"responseSchema":{"type":"json","example":{"ok":true,"title":"Example article title","robots":{"url":"https://www.example.com/robots.txt","found":true,"allowedForGenericBot":true},"status":200,"finalUrl":"https://www.example.com/article","warnings":[],"checkedAt":"2026-08-30T10:00:02.000Z","redirects":[{"to":"https://www.example.com/article","from":"https://example.com/article","status":301}],"likelyHtml":true,"contentType":"text/html; charset=utf-8","canonicalUrl":"https://www.example.com/article","requestedUrl":"https://example.com/article","contentLength":123456,"responseTimeMs":184}},"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.003","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.003/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.003","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.003","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_LE_sf6qyFTZCqD7ls5zg7","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.003","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Resolves redirects and returns deterministic HTTP facts (final URL, status, content type, robots.txt access, title) for a URL so an agent can decide whether to fetch it fully.","exampleAgentPrompt":"Before you go ahead and fetch that article at https://example.com/article, can you do a quick preflight check to see if it's accessible, what it resolves to, and whether bots are allowed?","exampleUseCases":[{"title":"Pre-crawl URL viability check","prompt":"Before you scrape that product page at https://shop.example.com/products/widget-pro, do a preflight check to confirm it's live, not paywalled, and that crawlers are permitted."},{"title":"Redirect chain investigation","prompt":"Can you trace where https://bit.ly/3xAbc12 actually ends up — I want to know the final URL, all the hops along the way, and the HTTP status at the end before I click anything."},{"title":"Bot access and content-type gating","prompt":"I'm about to have my agent download a bunch of PDFs from https://reports.example.org/annual-2024.pdf — first do a preflight to confirm it's really a PDF and that robots.txt won't block us."}],"resultDescription":"A JSON object containing: whether the request succeeded (ok), the final resolved URL, HTTP status code, content type and length, response time in ms, HTML page title, canonical URL, the full redirect chain with per-hop status codes, a robots.txt check result (found, URL, and whether generic bots are allowed), a timestamp, and any warnings encountered.","failureModes":["URL is unreachable or DNS fails — ok:false with network error details","Redirect loop detected — bounded hop limit stops resolution early with warning","URL returns non-HTML resource — likelyHtml:false, title absent","robots.txt not found — found:false in robots object","Malformed or excessively long URL — validation error before any fetch","Remote server rate-limits or blocks the preflight request — non-200 status returned"],"whenToPreferThis":"Use this endpoint whenever an agent is about to fetch an unfamiliar URL and wants to avoid wasting tokens, money, or crawler capacity on dead links, redirects to unexpected content, or bot-blocked resources. Prefer it over a full fetch when you only need HTTP metadata, redirect resolution, or robots.txt compliance status — not the full page body. It does not scrape page content, render JavaScript, or perform safety classification, so pair it with a scraping tool when full content is needed.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-14T00:33:52.370Z","isFirstParty":false}