{"uid":"cap_JNa4ysCV8OBJFToVxxdCC","slug":"screwtape-ai-judge-agent-as-a-service-b0ba4d7e","name":"Screwtape.ai — Judge Agent as a Service","description":"API for pass, fail, or retry verdicts on agent output, plus multi-agent consensus and false-consensus risk.","url":"https://screwtape.ai/v1/judge?utm_source=zero.xyz","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"task":{"type":"string"},"retain":{"type":"boolean"},"version":{"type":"string","maxLength":64,"description":"Optional version tag for that agent."},"agent_id":{"type":"string","pattern":"^[A-Za-z0-9._:-]{1,64}$","maxLength":64,"description":"Optional tag for the agent making this call."},"criteria":{"type":"array","items":{"type":"string"}},"evidence":{"type":"array","items":{"type":"string"}},"agent_output":{"type":"string"}}},"responseSchema":{"type":"json","schema":{"type":"object","required":["verdict","score","reasons","request_id"],"properties":{"score":{"type":"number"},"engine":{"type":"string"},"reasons":{"type":"array","items":{"type":"string"}},"verdict":{"enum":["pass","fail","retry"],"type":"string"},"retention":{"type":"object"},"request_id":{"type":"string"}}},"example":{"score":0.9,"checks":[],"claims":[{"text":"The crew size was 4.","status":"supported"},{"text":"The mission launched from Cape Canaveral.","status":"supported"}],"engine":"deterministic","reasons":["2 of 2 claims are grounded in the supplied evidence.","2 of 2 criteria matched the output.","Deterministic string checks only. No model was called."],"verdict":"pass","retention":{"retained":false},"request_id":"example-request-id","verification":"heuristic"}},"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.005","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.005/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.005","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.005","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_d27bCd93SXbbAGe39dnvu","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.005","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Evaluates agent output against a task and criteria, returning a pass/fail/retry verdict with a confidence score and reasons.","exampleAgentPrompt":"Judge this agent output for me: the task was 'Summarize the Apollo 11 mission', the output is 'Apollo 11 landed on the moon in July 1969 with a crew of 3 astronauts', and the criteria are ['crew size is accurate', 'date is correct'] — use the evidence 'Apollo 11 crew: Armstrong, Aldrin, Collins' and 'Launch date: July 16, 1969' to give me a pass, fail, or retry verdict with a score and reasons.","exampleUseCases":[{"title":"Automated QA gate for a research agent","prompt":"My research agent just produced this output: 'The Treaty of Versailles was signed in 1920 and ended World War I.' The task was 'Describe the Treaty of Versailles' and my criteria are ['correct year of signing', 'correct war it ended']. Use this evidence: 'The treaty was signed June 28, 1919' — can you judge whether this passes, fails, or needs a retry?"},{"title":"Content accuracy check for a writing agent","prompt":"I need you to evaluate my writing agent's output against the task 'Write a product description for our new wireless headphones' with criteria ['mentions noise cancellation', 'mentions battery life', 'no false claims'] and this evidence from our spec sheet: 'ANC enabled, 40hr battery'. The agent output was: 'Experience crystal-clear sound with active noise cancellation and a 30-hour battery.' Give me a verdict and score."},{"title":"Regression testing for a customer support bot","prompt":"Can you judge this support bot response for the task 'Answer: how do I reset my password?' with criteria ['mentions email link', 'steps are under 3 sentences', 'does not mention phone support'] and the agent output 'Click the reset link sent to your email, then follow the instructions to set a new password'? Tell me if it passes, fails, or should retry."}],"resultDescription":"Returns a JSON object containing a verdict enum (pass, fail, or retry), a numeric confidence score (0–1), an array of human-readable reasons explaining the verdict, a request_id for tracking, optional retention metadata if retain was set, and the engine type used (e.g. deterministic or model-based).","failureModes":["Missing required fields (task, agent_output, or criteria) result in a 400 error","Evidence array not grounded in the output may cause inaccurate heuristic scoring","Very long agent_output or criteria arrays may exceed token/processing limits","If criteria are ambiguous or contradictory, the verdict may be unreliable","Payment failure (x402) blocks the call entirely with a 402 response"],"whenToPreferThis":"Choose this endpoint when you need a structured, auditable pass/fail/retry verdict on LLM or agent output — especially when you have explicit criteria and supporting evidence to ground the evaluation. It is ideal for automated QA pipelines, agent self-evaluation loops, and regression testing where you need scored, reasoned verdicts rather than a raw similarity score or human review. Prefer it over prompt-based self-critique when you want deterministic, evidence-grounded checks with a machine-readable verdict enum.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-30T06:32:56.984Z","isFirstParty":false,"canonicalSlug":"screwtape-ai-judge-agent-as-a-service-b0ba4d7e"}