{"uid":"cap_mKdW5AkHB1x1GV2ICZRzl","slug":"arxiv-digest-agentdiet-dev-automated-llms-txt-agent-documentation-fc2d43b6","name":"Automated /llms.txt & Agent Documentation Context Compiler","description":"Crawls any documentation or SaaS site and compiles a standards-compliant /llms.txt manifest and JSON-LD schema bundle. (Live x402 v2, IETF MPP & MCP endpoint at https://arxiv-digest.agentdiet.dev/v1/discovery-manifest)","url":"https://arxiv-digest.agentdiet.dev/v1/discovery-manifest?utm_source=zero.xyz","method":"GET","headers":{},"bodySchema":{"type":"object","required":["url"],"properties":{"url":{"type":"string","minLength":1,"description":"Website URL to crawl and generate a ready-to-consume /llms.txt manifest for"}}},"responseSchema":{"type":"object"},"example":null,"exampleRequest":null,"tags":[],"displayCostAmount":"0.015","displayCostAsset":"USD","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.015/call","primary":{"kind":"static","protocol":"mpp","network":"tempo","amountUsd":"0.015","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"mpp","network":"tempo","amountUsd":"0.015","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_KFK8Fu_pVv1MOjgg4ZFXs","protocol":"mpp","methodType":"crypto","chain":"tempo","mode":"charge","costAmount":"0.015","costPer":"request","priority":0,"asset":"0x833589fcd6edb6e08f4c7c32d4f71b54bda02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":"arxiv-digest.agentdiet.dev","brandSlug":"arxiv-digest-agentdiet-dev-mpp","brandBaseUrl":"https://arxiv-digest.agentdiet.dev","brandDocsUrl":null,"whatItDoes":"Crawls any documentation or SaaS website and generates a standards-compliant /llms.txt manifest plus JSON-LD schema bundle for AI agent consumption","exampleAgentPrompt":"Crawl the documentation at https://docs.stripe.com and compile a standards-compliant /llms.txt manifest with a JSON-LD schema bundle so AI agents can discover and consume Stripe's API context.","exampleUseCases":[{"title":"Auto-generate llms.txt for SaaS docs","prompt":"Take my product documentation site at https://docs.myproduct.com and generate a ready-to-publish /llms.txt manifest so that AI agents can automatically discover what my platform does and how to use it."},{"title":"Make competitor docs agent-readable","prompt":"Crawl https://docs.notion.so and give me a structured /llms.txt manifest and JSON-LD schema bundle for Notion's documentation — I want to feed it into my agent as grounded context."},{"title":"Bootstrap AI context for internal tooling","prompt":"Generate an /llms.txt manifest from our internal developer portal at https://developer.acmecorp.internal so our agent fleet can auto-discover available tools and APIs without manual curation."}],"resultDescription":"Returns a standards-compliant /llms.txt manifest and JSON-LD schema bundle compiled from the crawled website, structured for direct consumption by AI agents, LLMs, and MCP-compatible toolchains.","failureModes":["URL is unreachable or returns non-200 status — crawl fails with error","Heavily JavaScript-rendered sites may yield incomplete context if JS execution is limited","Private or authentication-gated documentation cannot be crawled without credentials","Malformed or non-standard documentation structure may produce incomplete manifests","Rate limiting or robots.txt restrictions on the target site may block crawling"],"whenToPreferThis":"Use this endpoint when you need to automatically generate a machine-readable /llms.txt context file from any public documentation or SaaS site, especially when onboarding a new API or tool into an agent workflow and you want structured, standards-compliant agent context without manual curation. Prefer this over manual documentation parsing when speed and standards compliance (/llms.txt, JSON-LD) matter.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-21T07:19:25.318Z","isFirstParty":false,"canonicalSlug":"arxiv-digest-agentdiet-dev-automated-llms-txt-agent-documentation-0373c415"}