{"uid":"cap_R4_3rLT4wu3bhudmK7OlN","slug":"agent-data-bench-web-page-extractor-d68ea746","name":"Agent Data Bench – Web Page Extractor","description":"Turn a public web page into an LLM-ready document: clean markdown with nav/cookie/script boilerplate removed, parsed JSON-LD, OpenGraph, heading outline, tables, link graph, word and token counts, and embedding-ready chunks. Query: url (required), format=markdown, include=jsonld,headings,tables,links,chunks,openGraph. SSRF-guarded, 2MB cap, honours robots.txt.","url":"https://x402.deepsense.stream/api/v1/web/extract?utm_source=zero.xyz","method":"GET","headers":{},"bodySchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["input"],"properties":{"input":{"type":"object","required":["type","method"],"properties":{"type":{"type":"string","const":"http"},"method":{"enum":["GET"],"type":"string"},"queryParams":{"type":"object","required":["url"],"properties":{"url":{"type":"string","description":"Public http(s) URL"}}}},"additionalProperties":false},"output":{"type":"object","required":["type"],"properties":{"type":{"type":"string"},"example":{"type":"object"}}}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.005","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.005/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.005","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.005","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_uXgr7PKW0dnxN5rCFLg4N","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.005","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Fetches a public web page and returns LLM-ready content: clean markdown, JSON-LD, OpenGraph, heading outline, tables, link graph, word/token counts, and embedding-ready chunks.","exampleAgentPrompt":"Can you fetch https://en.wikipedia.org/wiki/Artificial_intelligence and give me the clean markdown content, heading outline, JSON-LD, and embedding-ready chunks so I can load it into my RAG pipeline?","exampleUseCases":[{"title":"RAG pipeline content ingestion","prompt":"Grab https://docs.openai.com/overview and extract the clean markdown and embedding-ready chunks — I want to load this page into my vector database."},{"title":"Structured metadata extraction","prompt":"Pull the JSON-LD and OpenGraph metadata from https://schema.org/Person so I can understand what structured data the page publishes."},{"title":"Research article outline extraction","prompt":"Fetch https://www.nature.com/articles/d41586-024-00001-0 and give me the heading outline and all the tables from the page so I can summarize the structure."}],"resultDescription":"Returns a JSON object containing: cleaned markdown of the page body (boilerplate removed), parsed JSON-LD objects, OpenGraph tags, hierarchical heading outline, extracted tables, a link graph of all page links, word count, token count, and chunked text segments ready for embedding — all derived from the fetched URL.","failureModes":["URL not publicly accessible or requires authentication — returns error","Target page blocked by robots.txt — request rejected per robots.txt compliance","Response body exceeds 2MB cap — truncated or error returned","SSRF guard triggered for private/internal IP ranges — request blocked","Malformed or non-HTTP URL provided — validation error","Target server unreachable or times out — network error returned","Page contains no parseable content (e.g. pure JS SPA with no SSR) — sparse or empty markdown"],"whenToPreferThis":"Choose this endpoint when you need to convert a single public web page into structured, LLM-ready content — especially when you need multiple data layers at once (clean markdown + JSON-LD + chunks + links) in one paid call. Prefer this over generic scraping tools when your goal is downstream LLM or RAG use, since it handles tokenization, chunking, and boilerplate removal natively. Avoid if you need to crawl multiple pages or require JavaScript rendering of client-side-only SPAs.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-20T12:35:31.184Z","isFirstParty":false,"canonicalSlug":"agent-data-bench-web-page-extractor-d68ea746"}