ab-test-plan
starsforks
How to install ab-test-plan?
indranilbanerjee/digital-marketing-pro/ab-test-plan
$npx -y skills add indranilbanerjee/digital-marketing-pro --skill ab-test-planDesign A/B and multivariate tests. Use when: sample size calculation, testing hypothesis, CRO experimentation.
SKILL.md
| 1 | # /digital-marketing-pro:ab-test-plan |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | Dedicated A/B test planning with a structured hypothesis framework, statistical sample size calculation, variant design, and monitoring plan. Produces a complete experiment specification with statistical rigor and clear decision criteria. |
| 6 | |
| 7 | ## Input Required |
| 8 | |
| 9 | The user must provide (or will be prompted for): |
| 10 | |
| 11 | - **Element to test**: The specific page, component, or experience being tested (landing page headline, CTA button, pricing page layout, email subject line, checkout flow, form design, etc.) |
| 12 | - **Current conversion rate**: Baseline conversion rate for the metric being tested (or best estimate) |
| 13 | - **Desired minimum detectable effect (MDE)**: The smallest improvement worth detecting. **MDE is ABSOLUTE by default** — expressed in the same units as the baseline (baseline 5.0% and you want to catch a +1.0 percentage-point lift, i.e. 5.0% → 6.0% ⇒ `--mde 0.01 --mde-type absolute`). To express it as a **relative** lift instead (a 10% relative improvement on a 5% baseline = 5.5% ⇒ `--mde 0.10 --mde-type relative`), pass `--mde-type relative`. This distinction is the single most common sample-size error: the same "10%" read as absolute vs. relative changes the required sample size by roughly 40× at a 5% baseline. Always confirm which the user means. |
| 14 | - **Daily traffic or impressions**: Average daily visitors or impressions to the test page or element |
| 15 | - **Significance level**: Desired confidence level, default 95% (alpha = 0.05) |
| 16 | - **Statistical power**: Desired power, default 80% (beta = 0.20) |
| 17 | - **Number of variants**: How many variants to test (default 1 treatment + 1 control; more for multivariate) |
| 18 | - **Business context**: What prompted the test idea (analytics data, user feedback, competitive analysis, heuristic audit, stakeholder request) |
| 19 | |
| 20 | ## Process |
| 21 | |
| 22 | 1. **Load brand context**: Read `~/.claude-marketing/brands/_active-brand.json` for the active slug, then load `~/.claude-marketing/brands/{slug}/profile.json`. Apply voice, compliance, industry context. Check `guidelines/_manifest.json` for restrictions, messaging, channel styles, voice-and-tone rules, and templates. If a template matching this command exists in `~/.claude-marketing/brands/{slug}/templates/`, apply its format. If no brand exists, prompt for `/digital-marketing-pro:brand-setup` or proceed with defaults. |
| 23 | 2. **Check campaign history**: Run `python "${CLAUDE_PLUGIN_ROOT}/scripts/campaign-tracker.py" --brand {slug} --action list-campaigns` to review past test results and avoid re-testing already-validated hypotheses. |
| 24 | 3. **Run sample size calculator**: Execute the calculator with the baseline rate, MDE, MDE type, significance, and power. The `--mde-type` flag defaults to `absolute` — always confirm with the user which interpretation they mean before computing (the two differ by ~40× at a 5% baseline): |
| 25 | ```bash |
| 26 | # Absolute MDE — detect a 1.0 percentage-point lift on a 5% baseline (5.0% → 6.0%) |
| 27 | python "${CLAUDE_PLUGIN_ROOT}/scripts/sample-size-calculator.py" --baseline-rate 0.05 --mde 0.01 --mde-type absolute --significance 0.95 --power 0.80 |
| 28 | |
| 29 | # Relative MDE — detect a 10% relative lift on a 5% baseline (5.0% → 5.5%) |
| 30 | python "${CLAUDE_PLUGIN_ROOT}/scripts/sample-size-calculator.py" --baseline-rate 0.05 --mde 0.10 --mde-type relative --significance 0.95 --power 0.80 |
| 31 | ``` |
| 32 | This determines the required sample size per variant. Later, when the test has run, evaluate the result with `python "${CLAUDE_PLUGIN_ROOT}/scripts/significance-tester.py" --control-visitors {n} --control-conversions {n} --variant-visitors {n} --variant-conversions {n} --confidence 0.95`. |
| 33 | 4. **Build hypothesis statement**: Structure the hypothesis in the format: "If [specific change], then [primary metric] will [direction and magnitude] because [rationale grounded in data, user research, or established UX principle]." |
| 34 | 5. **Design test variants**: Define the control (current experience) and one or more treatment variants. Specify exactly what changes in each variant -- copy, layout, color, imagery, flow, or functionality. For multivariate tests, define the variable matrix and interaction effects to watch. |
| 35 | 6. **Define primary and secondary metrics**: Identify the primary success metric (the one that determines the winner) and secondary metrics to monitor for unintended effects (e.g., testing CTA click rate as primary, but watching bounce rate, time on page, and downstream conversion as secondary guardrails). |
| 36 | 7. **Calculate test duration**: Based on sample size requirements and daily traffic, estimate the number of days needed. Ensure the duration spans at least one full business cycle (7 days minimum) to account for day-of-week variation. Flag if duration exceeds 8 weeks (validity risk). |
| 37 | 8. **Create monitoring plan**: Define interim checkpoin |
People who liked this also viewed
6 picksType
- seo-auditWhen the user wants to audit, review, or diagnose SEO issues on their site. Also use when the user mentions "SEO audit," "technical SEO," "why am I not ranking," "SEO issues," "on-page SEO," "meta tagSkillsJul 2026
- copywritingWhen the user wants to write, rewrite, or improve marketing copy for any page — including homepage, landing pages, pricing pages, feature pages, about pages, or product pages. Also use when the user sSkillsJul 2026
- marketing-psychologyWhen the user wants to apply psychological principles, mental models, or behavioral science to marketing. Also use when the user mentions 'psychology,' 'mental models,' 'cognitive bias,' 'persuasion,'SkillsJul 2026
- content-strategyWhen the user wants to plan a content strategy, decide what content to create, or figure out what topics to cover. Also use when the user mentions "content strategy," "what should I write about," "conSkillsJul 2026
- programmatic-seoWhen the user wants to create SEO-driven pages at scale using templates and data. Also use when the user mentions "programmatic SEO," "template pages," "pages at scale," "directory pages," "location pSkillsJul 2026
- marketing-ideasWhen the user needs marketing ideas, inspiration, or strategies for their SaaS or software product. Also use when the user asks for 'marketing ideas,' 'growth ideas,' 'how to market,' 'marketing stratSkillsJul 2026
