{"uid":"cap_zAEqblBe1eG2T15cE6n8d","slug":"aialign-capability-evaluation-weighted-pass-rate-score-aggregator-0f16e17f","name":"AIAlign Capability Evaluation – Weighted Pass Rate & Score Aggregator","description":"Computes weighted pass rates and weighted scores from caller-labeled test outcomes, with deterministic category summaries. It does not run tests, validate labels, measure untested capabilities or establish deployment safety; the result is only as representative as the supplied evaluation set.","url":"https://aialign.halowerk.com/v1/capability-eval","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"test_cases":{"type":"array","items":{"type":"object","required":["case_id","category","passed","score","weight"],"properties":{"score":{"type":"number","maximum":1,"minimum":0},"passed":{"type":"boolean"},"weight":{"type":"number","maximum":1000000,"minimum":0.000001},"case_id":{"type":"string","maxLength":128,"minLength":1},"category":{"type":"string","maxLength":128,"minLength":1}},"additionalProperties":false},"maxItems":100000,"minItems":1}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.003","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.003/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.003","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.003","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_5tPA5oWOstXmybziOcSEU","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.003","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Aggregates caller-labeled test cases into weighted pass rates and weighted scores with deterministic per-category summaries for AI capability evaluation.","exampleAgentPrompt":"I have 500 labeled test cases for our AI assistant — each one has a category like 'reasoning' or 'safety', a pass/fail result, a score between 0 and 1, and a weight up to 1000 — can you compute the weighted pass rates and weighted scores broken down by category?","exampleUseCases":[{"title":"LLM benchmark rollup by capability","prompt":"We ran 2,000 test cases across categories like 'math', 'coding', and 'instruction-following', each with a pass/fail flag, a 0–1 score, and a weight reflecting test importance — can you compute the weighted pass rate and weighted score for each category so I can see where the model is weakest?"},{"title":"Safety evaluation weighted summary","prompt":"I have a set of safety eval test cases tagged as 'harmful-content', 'privacy', or 'bias', each with a pass/fail label, a numeric score, and a weight. Please aggregate them into weighted pass rates and category-level summaries so I can report the overall safety posture."},{"title":"Regression testing score comparison prep","prompt":"After our latest model update I labeled 300 regression test cases across three feature categories with their pass/fail outcomes, scores, and weights — compute the weighted scores and pass rates per category so I can compare against last sprint's baseline."}],"resultDescription":"Returns weighted pass rates and weighted aggregate scores per category, computed deterministically from the supplied test case array. Each category entry reflects the sum-of-weights-normalized pass rate and score across all test cases tagged to that category. The result is purely a function of the submitted data; no external inference or validation is performed.","failureModes":["Missing required fields (case_id, category, passed, score, weight) cause validation errors","Weight values outside [0.000001, 1000000] or scores outside [0, 1] are rejected","Empty test_cases array or array exceeding 100,000 items returns a bounds error","Malformed JSON body returns a 400 parse error","Payment failure (x402) returns a 402 response before processing begins"],"whenToPreferThis":"Choose this endpoint when you already have labeled test outcomes and want a fast, deterministic, weighted aggregation by category — not when you need to actually run tests, validate labels, or assess untested capabilities. It is ideal for post-evaluation rollup pipelines where the evaluation set is fixed and you need reproducible per-category pass rate and score summaries without any ML inference overhead.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-14T06:39:56.312Z","isFirstParty":false}