{"uid":"cap_G1fnWMgVCWkeZE87hKXte","slug":"synthex-forge-mcp-eval-judge-196e21ed","name":"Synthex Forge MCP Eval Judge","description":"Synthex Forge x402 Receive: 41 paid verification, attestation and governance endpoints (Ed25519 + SHA-256 signed artifacts) plus free verify/public-key/standard sibling routes. Paid routes answer HTTP 402 with an x402 v2 `payment-required` envelope (atomic USDC on Base mainnet, chain ID 8453). Free routes are declared with `security: []` per the x402scan discovery spec.","url":"https://x402.synthexforge.com/v1/mcp/eval/judge?utm_source=zero.xyz","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"rubric":{"type":"array","items":{"type":"object"},"description":"Rubric: [{criterion?, kind?: token-overlap|exact-match|fact-presence|length-window|keyword-presence, weight?, target?, min_len?, max_len?}]"},"subject":{"type":"string","description":"Optional human label for the eval job / batch."},"response_pair":{"type":"object","description":"{candidate, reference?, question?, model?, source?} — the response pair to grade."}}},"responseSchema":{"type":"json","example":{"ok":true}},"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.15","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.15/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.15","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.15","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_2zpBPlQ1fg1blLlvzWL__","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.15","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Grades a candidate LLM response against a multi-criterion rubric, returning a scored evaluation artifact signed with Ed25519 and SHA-256 on-chain via x402","exampleAgentPrompt":"Grade this model response against my rubric: rubric has two criteria — fact-presence (weight 0.6, target 'Paris is the capital of France') and length-window (weight 0.4, min_len 20, max_len 200) — and the candidate answer is 'Paris, the capital of France, is known for the Eiffel Tower.' with reference 'Paris is the capital of France.' Label the job 'capital-city-eval'.","exampleUseCases":[{"title":"Automated RAG answer quality check","prompt":"I need to grade my RAG pipeline's answer to the question 'What caused the 2008 financial crisis?' — the candidate says 'Subprime mortgage lending and excessive leverage caused the crisis.' and the reference is 'The 2008 crisis was driven by subprime mortgage failures and systemic leverage.' Use a rubric with token-overlap (weight 0.7) and length-window between 10 and 80 words (weight 0.3), and label the job 'rag-finance-eval'."},{"title":"LLM benchmark grading with attestation","prompt":"Run a rubric eval on my benchmark results: the question is 'Name the planets in the solar system', the candidate response lists all 8 planets correctly, the reference lists all 8, use a fact-presence criterion checking for 'Neptune' and 'Mercury' each at weight 0.5, and call the subject 'planet-benchmark-run-1'. I need the signed result for governance records."},{"title":"Keyword compliance check for policy outputs","prompt":"Judge this AI-generated policy summary against a keyword-presence rubric — criteria: keyword-presence for 'GDPR' (weight 0.4), keyword-presence for 'data subject' (weight 0.3), and exact-match for 'Article 17' (weight 0.3). The candidate text is 'Under GDPR, data subject rights include erasure under Article 17.' Label this eval 'policy-compliance-check'."}],"resultDescription":"A JSON object containing an overall pass/fail flag (ok: true/false), per-criterion scores and weights, an aggregated weighted score, and a cryptographically signed Ed25519 + SHA-256 attestation artifact anchored on Base mainnet (chain ID 8453) for auditability and governance.","failureModes":["Missing required response_pair field returns a 400 validation error","Malformed rubric array (missing kind or weight) may produce partial scores or a 422","Payment not settled causes HTTP 402 with x402 v2 payment-required envelope before processing","Invalid or unsupported criterion kind (e.g. typo) may cause criterion to be skipped or error","Empty candidate text may yield a zero score without error","Network or chain issues may delay attestation signing"],"whenToPreferThis":"Choose this endpoint when you need a cryptographically attested, rubric-based evaluation of LLM outputs — especially for governance, compliance audit trails, or multi-criterion grading where the signed artifact matters. Prefer it over plain text diff tools or simple similarity APIs when you need provable, on-chain attestation of the eval result, or when you require flexible criterion types (token-overlap, exact-match, fact-presence, length-window, keyword-presence) in a single call.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-23T01:13:25.728Z","isFirstParty":false,"canonicalSlug":"synthex-forge-mcp-eval-judge-196e21ed"}