ab-test-analysis
Analyze A/B test results with statistical significance, sample size validation
by phurynFree
Description
ab-test-analysis is an agent skill for evaluating A/B test results with statistical rigor and turning the findings into a product decision. It checks statistical significance, validates the sample size, calculates confidence intervals, and recommends whether to ship the variant, extend the test, or stop it. Before it analyzes anything, it works through the experiment: the hypothesis, the change in the variant, the primary metric and any guardrail metrics, how long the test ran, and the traffic split. If you provide CSV, Excel or analytics export files, it reads them directly and can generate Python scripts for the statistical calculations. The skill is for product managers and analysts who need a clear answer from an experiment, rather than a headline percentage that may not be meaningful. It is part of the pm-skills collection by phuryn and is listed on skills.sh as free.…
Highlights
- Checks statistical significance and sample size
- Calculates confidence intervals
- Recommends ship, extend or stop
- Reads CSV, Excel or analytics exports directly
Ratings & Reviews
No ratings yet
Be the first to rate ab-test-analysis.
People also visit
brainstorm-experiments-existing
Design experiments to test assumptions for an existing product
metrics-review
Review and analyze product metrics with trend analysis and actionable insights
statistical-analysis
Apply statistical methods including descriptive stats, trend analysis
validate-data
QA an analysis before sharing -- methodology, accuracy, and bias checks
ab-test-analysis FAQ
What does ab-test-analysis decide?
Whether to ship the variant, extend the test, or stop it, based on the statistics and the experiment setup.
What does it need from me?
The hypothesis, the variant, the primary and guardrail metrics, the run length, the traffic split, and your data if you have it.
Can it work from my data file?
Yes. It reads CSV, Excel and analytics exports directly.
Who is it for?
Product managers and analysts reviewing experiment results.
How do I install it?
Run npx skills add https://github.com/phuryn/pm-skills --skill ab-test-analysis in your terminal.
Is it free?
The listing shows the skill as free.
What are the best ab-test-analysis alternatives?
Popular alternatives to ab-test-analysis include brainstorm-experiments-existing, metrics-review and statistical-analysis. See all ab-test-analysis alternatives