{"uid":"cap_MGJnzVBZbCcw5s_MKA2hm","slug":"x402-seller-web-scrape-js-cloudflare-bypass-09307d7a","name":"x402-seller Web Scrape (JS/Cloudflare Bypass)","description":"Page-to-Markdown extraction for hard sites — JS-rendered pages and, in most tested cases, active Cloudflare challenges (verified: 3 of 4 real sites bypassed cleanly). A second engine alongside /api/web/read, better for a clean single article. JSON body: {url: string (http/https, publicly reachable)}.","url":"https://x402-seller-0ay3.onrender.com/api/web/scrape","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"url":{"type":"string","description":"Public http(s) URL of the page to extract."}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.02","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.02/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_Sz8dQXgmLTIcHGQ4CQ85U","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.02","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Fetches a publicly reachable URL and returns its full content as clean Markdown, including JS-rendered pages and most Cloudflare-protected sites.","exampleAgentPrompt":"Can you scrape the article at https://example.com/some-article and give me the full content as clean Markdown? The page is JavaScript-rendered and may be behind Cloudflare.","exampleUseCases":[{"title":"Reading a paywalled JS-rendered article","prompt":"Can you extract the full article text from https://techcrunch.com/2024/05/01/some-story as Markdown? The page uses heavy JavaScript and I can't get the content with a regular fetch."},{"title":"Bypassing Cloudflare to read a product page","prompt":"Go to https://some-protected-shop.com/product/widget and pull back all the visible text as Markdown — the site has Cloudflare protection so a normal request won't work."},{"title":"Archiving a news page for analysis","prompt":"Scrape https://reuters.com/article/some-news-piece and return its full content in Markdown so I can analyze the text and pull out key facts."}],"resultDescription":"A JSON response containing the extracted page content rendered as clean Markdown text, preserving headings, paragraphs, and links from the original page, even if it was JavaScript-rendered or protected by Cloudflare.","failureModes":["Cloudflare challenge not bypassed (roughly 1 in 4 real Cloudflare sites may fail) — returns an error or empty/blocked content","Invalid or non-public URL returns a 4xx error","Timeout on very slow or unresponsive pages","Paywalled content behind login walls cannot be accessed — only publicly reachable pages work","Malformed URL in request body returns a validation error"],"whenToPreferThis":"Prefer this endpoint over simpler fetch/read endpoints when the target page uses heavy JavaScript rendering or is behind a Cloudflare anti-bot challenge. It is optimized for single-article or single-page extraction producing clean Markdown, making it ideal when you need readable prose rather than raw HTML. Choose the sibling /api/web/read endpoint if the page is a plain static site, since this endpoint costs $0.02 per call and is best reserved for harder targets.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-13T18:44:09.325Z","isFirstParty":false}