{"uid":"cap__ZBUEqQz2eILeO1ffKknR","slug":"a-b-test-statistical-significance-calculator-1d4da3c8","name":"A/B Test Statistical Significance Calculator","description":"A/B conversion test: rates, relative lift, two-proportion z-test p-value, confidence interval and a significance verdict.","url":"https://math-stats.x402tookit.com/ab-test?utm_source=zero.xyz","method":"GET","headers":{},"bodySchema":{"type":"object","$schema":"https://json-schema.org/draft/2020-12/schema","required":["input"],"properties":{"input":{"type":"object","required":["type","method"],"properties":{"type":{"type":"string","const":"http"},"method":{"enum":["GET"],"type":"string"},"queryParams":{"type":"object","required":["visitorsA","conversionsA","visitorsB","conversionsB"],"properties":{"alpha":{"type":"number","description":"Significance level 0.001 to 0.2 (default 0.05)"},"visitorsA":{"type":"number","description":"Visitors in A (control)"},"visitorsB":{"type":"number","description":"Visitors in B (variant)"},"conversionsA":{"type":"number","description":"Conversions in A"},"conversionsB":{"type":"number","description":"Conversions in B"}}}},"additionalProperties":false},"output":{"type":"object","required":["type"],"properties":{"type":{"type":"string"},"example":{"type":"object","properties":{"asOf":{"type":["null","string"]},"alpha":{"type":["null","number"]},"rateA":{"type":["null","number"]},"rateB":{"type":["null","number"]},"source":{"type":["null","string"]},"reasons":{"type":["array","null"],"items":{"type":"string"}},"summary":{"type":["null","string"]},"verdict":{"type":["null","string"]},"degraded":{"type":["boolean","null"]},"disclaimer":{"type":["null","string"]},"notChecked":{"type":["array","null"],"items":{"type":"string"}},"pooledRate":{"type":["null","number"]},"zStatistic":{"type":["null","number"]},"significant":{"type":["boolean","null"]},"pValueTwoSided":{"type":["null","number"]},"relativeLiftPct":{"type":["null","number"]},"differencePoints":{"type":["null","number"]},"enoughConversions":{"type":["boolean","null"]},"confidenceIntervalPoints":{"type":["null","object"],"properties":{"level":{"type":["null","number"]},"lower":{"type":["null","number"]},"upper":{"type":["null","number"]}}}}}}}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.02","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.02/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_Nxq2fhBZ2VJoZ7SJ3_RZU","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.02","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Computes conversion rates, relative lift, two-proportion z-test p-value, confidence interval, and a significance verdict for an A/B experiment.","exampleAgentPrompt":"Can you run an A/B significance test on my experiment? Control had 10,000 visitors and 320 conversions, variant had 10,200 visitors and 380 conversions — use a 0.05 significance level and tell me if the lift is statistically significant.","exampleUseCases":[{"title":"Landing page conversion test","prompt":"My landing page A/B test just ended — control got 8,500 visitors and 255 conversions, the new design got 8,300 visitors and 299 conversions. Was the improvement statistically significant at the 0.05 level, and what's the relative lift?"},{"title":"Email campaign subject line experiment","prompt":"I ran a subject line test on my email campaign: version A had 25,000 opens out of 200,000 sends, version B had 27,800 opens out of 200,000 sends. Can you check if that difference is statistically significant using a 0.01 alpha?"},{"title":"Pricing page CTA button test","prompt":"We changed our CTA button color and want to know if it worked — the old button saw 450 conversions from 15,000 visitors, the new one got 510 conversions from 14,800 visitors. Run the two-proportion z-test and give me the p-value and confidence interval."}],"resultDescription":"Returns conversion rates for both groups, the absolute difference in percentage points, relative lift percentage, pooled rate, z-statistic, two-sided p-value, confidence interval (lower/upper bounds with level), a boolean significance flag, a human-readable verdict string, a summary, reasons list, whether there were enough conversions, and optional degraded/disclaimer fields.","failureModes":["Missing required parameters (visitorsA, conversionsA, visitorsB, conversionsB) returns a validation error","Alpha outside 0.001–0.2 range may be rejected or defaulted","Conversions exceeding visitor count produces invalid input error","Very small sample sizes may return a result with enoughConversions=false and a degraded flag","Zero visitors in either group causes division-by-zero handling with a degraded response"],"whenToPreferThis":"Choose this endpoint when you need a complete, ready-to-interpret A/B test report — not just a raw p-value — including relative lift, confidence intervals, and a plain-language verdict. It is ideal for product and marketing teams running two-group conversion experiments where a two-proportion z-test is the standard method. Prefer it over a generic statistics library when you want the verdict and summary text included in the response without post-processing.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-10-29T13:15:03.251Z","isFirstParty":false,"canonicalSlug":"a-b-test-statistical-significance-calculator-1d4da3c8"}