MarketingCollectible prompt

A/B test plan with a real hypothesis and sample size

Plan one A/B test properly: hypothesis, metric, sample size, run time and a decision rule set in advance.

ClaudeChatGPTGemini

Curated by Nvoka

Nvoka wordmark logo
Nvoka wordmark logoMarketing

A/B test plan with a real hypothesis and sample size

Act as an experimentation lead. Plan one A/B test on {{page_or_feature}} of {{website_url}}. The goal: {{goal}}. The change I want to test, and why I think it will help: {{test_idea}} 1. Hypothesis, in this form: because we saw a specific observation in data or research, we believe a specific change for a specific audience will move a specific metric, and we'll know at the planned sample size. If my reason is a hunch, say so and suggest the fastest way to get evidence first. 2. Metrics: one primary metric tied to the goal (completed bookings, for example, not button clicks); two or three guardrails that must not get worse, such as refunds, bounce rate or page speed; and how each is measured. 3. Sample size: ask for this page's current conversion rate and weekly visitors, then calculate visitors per variant for the smallest lift worth acting on, at 95% confidence and 80% power. Show the math and the expected run time. Run whole weeks, at least two, to cover weekday patterns. If traffic can't finish it in about six weeks, say so plainly and suggest a bolder change, a before-and-after comparison with caveats, or user testing instead. 4. Setup: a 50/50 split, sticky per visitor, no flicker (assign on the server or at the edge if possible), consent respected, and a sample ratio check to confirm the split works. 5. Decision rule, written before launch: no peeking and stopping early on a good day; what we do if it wins, loses or ties; and which segments we'll check, such as mobile versus desktop, without fishing for a winner. Return a one-page test plan I can share, with a results template.
Marketingnvoka.com/library/nvoka-ab-test-plan-with-a-hypothesis-and-sample-sizeScan to open
Marketing

A/B test plan with a real hypothesis and sample size

Plan one A/B test properly: hypothesis, metric, sample size, run time and a decision rule set in advance.

Nvoka logo

Curated by Nvoka

ClaudeChatGPTGeminiClaude Code
Add to my library

Make it yours

Fill in the blanks and change any word. Only your copy changes, never the card.

Fill in the blanks

0 of 4 filled

Your prompt

Act as an experimentation lead. Plan one A/B test on {{page_or_feature}} of {{website_url}}. The goal: {{goal}}. The change I want to test, and why I think it will help: {{test_idea}} 1. Hypothesis, in this form: because we saw a specific observation in data or research, we believe a specific change for a specific audience will move a specific metric, and we'll know at the planned sample size. If my reason is a hunch, say so and suggest the fastest way to get evidence first. 2. Metrics: one primary metric tied to the goal (completed bookings, for example, not button clicks); two or three guardrails that must not get worse, such as refunds, bounce rate or page speed; and how each is measured. 3. Sample size: ask for this page's current conversion rate and weekly visitors, then calculate visitors per variant for the smallest lift worth acting on, at 95% confidence and 80% power. Show the math and the expected run time. Run whole weeks, at least two, to cover weekday patterns. If traffic can't finish it in about six weeks, say so plainly and suggest a bolder change, a before-and-after comparison with caveats, or user testing instead. 4. Setup: a 50/50 split, sticky per visitor, no flicker (assign on the server or at the edge if possible), consent respected, and a sample ratio check to confirm the split works. 5. Decision rule, written before launch: no peeking and stopping early on a good day; what we do if it wins, loses or ties; and which segments we'll check, such as mobile versus desktop, without fishing for a winner. Return a one-page test plan I can share, with a results template.
See all