{"uid":"cap_IlWtfIXUOqRjiLTN8WFmo","slug":"a-b-test-significance-calculator-two-proportion-z-test-9eb400e6","name":"A/B Test Significance Calculator (Two-Proportion Z-Test)","description":"A/B test significance with a two-proportion z-test: observed lift, z-score, two-sided p-value, confidence interval on the lift, minimum detectable effect, and sample-size sufficiency. Pure computation over your inputs.","url":"https://api.agentstools.dev/marketing/experiment","method":"POST","headers":{},"bodySchema":{"type":"object","properties":{"a":{"type":"object","description":"Control arm with conversions and visitors"},"b":{"type":"object","description":"Variant arm with conversions and visitors"},"alpha":{"type":"number","description":"Significance level, default 0.05"},"power":{"type":"number","description":"Target power for sample-size math, default 0.8"}}},"responseSchema":null,"example":null,"exampleRequest":null,"tags":["x402"],"displayCostAmount":"0.02","displayCostAsset":"USDC","priceDynamic":false,"priceHint":null,"priceStatus":"priced","priceSource":"probe","requiresHandshake":false,"reviewCount":0,"rating":{"score":"0.00","successRate":"0.00","reviews":0,"stars":null,"state":"unrated"},"availabilityStatus":"unknown","priceObserved":null,"sessionDeposit":null,"pricing":{"kind":"static","summary":"$0.02/call","primary":{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"},"accepted":[{"kind":"static","protocol":"x402","network":"base","amountUsd":"0.02","per":"call","confidence":"exact"}]},"paymentMethods":[{"uid":"pm_OyGKQ_3YQOXb-hnsC8Xsb","protocol":"x402","methodType":"crypto","chain":"base","mode":"charge","costAmount":"0.02","costPer":"request","priority":0,"asset":"0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913","unit":"request","depositMicros":null,"planRef":null}],"brandName":null,"brandSlug":null,"brandBaseUrl":null,"brandDocsUrl":null,"whatItDoes":"Computes A/B test statistical significance using a two-proportion z-test, returning observed lift, z-score, two-sided p-value, confidence interval on the lift, minimum detectable effect, and sample-size sufficiency.","exampleAgentPrompt":"My control group had 5,200 visitors with 312 conversions, and my variant had 5,100 visitors with 357 conversions — run a two-proportion z-test at 95% confidence and tell me if the lift is statistically significant, what the p-value is, and whether my sample size is sufficient.","exampleUseCases":[{"title":"Email subject line A/B test readout","prompt":"I ran an email campaign split test: version A was sent to 8,000 people and got 640 opens, version B went to 7,950 people and got 730 opens. Can you run the two-proportion z-test at 95% confidence and tell me if B's higher open rate is statistically significant and what the observed lift and p-value are?"},{"title":"Landing page conversion experiment","prompt":"My landing page control had 12,400 visitors and 496 sign-ups, and the new variant had 11,800 visitors and 531 sign-ups. Check if that difference is statistically significant at 99% confidence, give me the z-score, p-value, and confidence interval on the lift, and tell me if my sample size is sufficient."},{"title":"Checking MDE before launching experiment","prompt":"I'm planning a pricing page test where my baseline conversion rate is about 3.2% and I expect roughly 4,000 visitors per arm over two weeks — what's the minimum detectable effect I can reliably measure at 95% confidence with that sample size?"}],"resultDescription":"Returns a JSON object containing: the observed lift (percentage difference between variant and control conversion rates), the z-score, a two-sided p-value, a confidence interval (lower and upper bounds) on the lift, the minimum detectable effect given the sample sizes, and a boolean or flag indicating whether the sample size is sufficient to draw reliable conclusions.","failureModes":["Missing or zero sample sizes cause division errors or invalid z-score computation","Conversion counts exceeding sample size return a validation error","Extremely small sample sizes may produce unreliable MDE or power estimates","Invalid confidence level values (outside 0–1 or not a recognized percentage) return an error","Network timeout or service unavailability returns a 5xx error"],"whenToPreferThis":"Choose this endpoint when you need a fast, pure-computation statistical significance check for a two-group proportion experiment (e.g. conversion rates, click-through rates) without setting up a local stats library. It is ideal for AI agents that need to interpret A/B test results on the fly, check sample-size sufficiency before declaring a winner, or compute MDE for experiment planning — all with no side effects and deterministic results from your inputs alone.","instructions":null,"reviewSummary":null,"reviewSummaryHighlights":null,"reviewSummaryConcerns":null,"reviewSummaryGeneratedAt":null,"activationCount":0,"lastUsedAt":null,"lastSuccessfullyRanAt":null,"lastHealthCheckAt":"2026-09-14T18:54:54.968Z","isFirstParty":false}