{"uid":"cap_UW9Pnt2c8BCOX-LBKAkUm","slug":"web-page-content-extractor-live-fetch-1c2cc258","name":"Web Page Content Extractor (Live Fetch)","description":"One maintained, normalized jobs data API that scrapes open job postings and listings across 10 applicant-tracking systems (Greenhouse, Lever, Ashby, SmartRecruiters, Workday, Recruitee, Workable, BambooHR, Breezy, Personio) with company-domain auto-detection. Use it to list a company’s open roles, find remote software-engineering / sales / any jobs, detect whether a company is hiring, and pull recruiting, salary and requirements data. Pay per call in USDC on Base via x402 (HTTP 402) — no account, no API key. Adapters are drift-monitored and maintained so the feed does not rot.","url":"https://assetfactory-jobs-x402.sigrunner.workers.dev/web/extract?utm_source=zero.xyz","method":"GET","headers":{},"bodySchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["input"],"properties":{"input":{"type":"object","required":["type","method"],"properties":{"type":{"type":"string","const":"http"},"method":{"enum":["GET"],"type":"string"},"queryParams":{"type":"object","required":["urls"],"properties":{"urls":{"type":"string","description":"Comma-separated page URLs to fetch and clean (up to 10). Each is fetched LIVE at request time (never a cached snapshot), boilerplate-stripped, and returned as clean main-content text + light Markdown + metadata + links. The \"read a live page before reasoning\" primitive for an agent."},"maxChars":{"type":"integer","default":40000,"maximum":200000,"description":"Per-URL ceiling on returned text / markdown length."},"includeLinks":{"type":"boolean","default":true,"description":"Return absolute outbound links (url + anchor text) from the main content."},"includeMarkdown":{"type":"boolean","default":true,"description":"Return a light Markdown rendering of the main content."}}}},"additionalProperties":false},"output":{"type":"object","required":["type"],"properties":{"type":{"type":"string"},"example":{"type":"object"}}}}},"responseSchema":{"type":"json","example":{"count":1,"stats":{"ok":1,"failed":0,"extracted":1,"requested":1},"records":[{"ok":true,"url":"https://blog.cloudflare.com/","kind":"html","lang":"en","text":"October 30, 2026 ... clean main-content prose ...","links":[{"url":"https://blog.cloudflare.com/post/","text":"Read more"}],"title":"Cloudflare Blog","author":null,"status":200,"headings":[{"text":"What took the Internet offline in Q3 2026","level":2}],"markdown":"## What took the Internet offline in Q3 2026\n\n...","canonical":"https://blog.cloudflare.com/","final_url":"https://blog.cloudflare.com/","framework":"static","published":null,"site_name":"The Cloudflare Blog","truncated":false,"char_count":6188,"structured":{"jsonld":[{"name":"What took the Internet offline","type":"Article"}],"jsonld_count":1},"word_count":858,"description":"Technical deep dives, product updates, and insights...","content_type":"text/html; charset=utf-8","recovered_from":null,"reading_time_min":4},{"ok":true,"url":"https://jobs.ashbyhq.com/openai","kind":"html","text":"Account Associate — Singapore ... $200K–$261K • Offers Equity ...","title":"OpenAI Jobs","status":200,"framework":"static","truncated":false,"word_count":2190,"recovered_from":"inline_state"}],"generated_at":"2026-10-30T00:00:00.000Z"}},"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.003","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.003/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.003","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.003","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_OejPDWA-6vf0o-IBlBcTX","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.003","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Fetches up to 10 public URLs live at request time, strips boilerplate, and returns clean main-content text, light Markdown, page metadata, and outbound links.","exampleAgentPrompt":"Fetch the live content from https://blog.cloudflare.com/ and https://news.ycombinator.com/ — strip all the boilerplate and give me the clean main-content text plus any outbound links, capped at 40000 characters per page.","exampleUseCases":[{"title":"Read article before answering question","prompt":"Before you answer my question about Anthropic's latest model, go fetch and read the live content at https://www.anthropic.com/news/claude-3-5-sonnet so you have the actual current text, not your training data."},{"title":"Competitive landing page analysis","prompt":"Fetch the live pages at https://stripe.com, https://braintree.com, and https://adyen.com — strip the nav and footer boilerplate and return the clean main-content text plus all outbound links from each page so I can compare their messaging."},{"title":"Extract blog post metadata for SEO audit","prompt":"Pull the live content from these five blog posts — https://example.com/post1, https://example.com/post2, https://example.com/post3 — and give me the title, description, canonical URL, author, published date, and word count for each one."}],"resultDescription":"Returns a JSON array of records, one per URL, each containing: ok status, cleaned main-content text, light Markdown rendering, page title, description, author, canonical URL, language, published date, site name, headings list, outbound links (URL + anchor text), HTTP status, final URL after redirects, char count, word count, reading time in minutes, and a truncated flag. Also includes aggregate stats (ok/failed/extracted counts) and a generated_at timestamp.","failureModes":["URL is not publicly accessible or returns a non-200 status — record has ok:false and status code","Page is behind a login/paywall — returns whatever the unauthenticated response contains, often a login page","SSRF-guarded: internal/private IP ranges and localhost URLs are rejected","More than 10 URLs provided — request is rejected or truncated","maxChars exceeded — content is truncated with truncated:true flag","Page renders content entirely via client-side JavaScript — static fetch may miss dynamic content","Network timeout on slow-responding origin — record marked failed"],"whenToPreferThis":"Choose this endpoint when an agent needs to read the actual live content of a public web page before reasoning — e.g. fact-checking, summarization, competitive research, or link extraction — and you want boilerplate-stripped clean text plus metadata in a single keyless call. Prefer this over cached/indexed search results when freshness matters. Supports batch fetching of up to 10 URLs per call, making it efficient for multi-page workflows.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-31T18:30:17.403Z","isFirstParty":false,"canonicalSlug":"web-page-content-extractor-live-fetch-1c2cc258"}