Growth Experiment Designer

90-Day Growth Experiment Example:

Hypothesis: Adding social proof to pricing page will increase trial signups by 15%
Metric: Trial signup rate (baseline: 3.2%)
Test Design: A/B split (50/50) for 4 weeks, min 1000 visitors per variant
Success Criteria: Statistical significance (95% confidence) + 15% lift

Progress:

• Define growth problem and current baseline metrics
• Generate testable hypotheses using ICE scoring (Impact/Confidence/Ease)
• Design experiment with clear variables and controls
• Set success criteria and statistical requirements
• Build tracking and measurement system
• Execute test for predetermined duration
• Analyze results and extract learnings
• Scale winning variants or iterate on failures

Hypothesis Framework:

1. Problem: What specific growth bottleneck are we addressing?
2. Hypothesis: "We believe that [change] will result in [outcome] because [assumption]"
3. Variables: Independent (what we change) vs dependent (what we measure)
4. Success Metrics: Primary KPI + supporting metrics + guardrails

Example 1: Email Onboarding Input: SaaS with 40% trial-to-paid conversion, want to improve activation Output:

• Hypothesis: Personalized onboarding email sequence will increase day-7 activation by 25%
• Test: 3 variants (control, personalized, interactive) over 6 weeks
• Metrics: Day-7 activation rate, trial-to-paid conversion, time-to-first-value

Example 2: Pricing Page Optimization Input: B2B service with 2.1% visitor-to-demo conversion rate Output:

• Hypothesis: Adding calculator tool will increase demo requests by 30%
• Test: Original vs calculator-enabled page, 4-week duration
• Metrics: Demo request rate, qualified lead rate, page engagement time

• ICE Scoring: Rate ideas on Impact (1-10), Confidence (1-10), Ease (1-10)
• Statistical Power: Ensure minimum sample size for 95% confidence, 80% power
• Test Duration: Run until statistical significance OR predetermined time limit
• Segmentation: Analyze results by customer segments, traffic sources, devices
• Learning Documentation: Record both wins and failures for future reference
• Velocity: Aim for 2-4 experiments running simultaneously in different funnels

• Testing too many variables simultaneously (stick to single variable tests)
• stopping tests early due to impatience (wait for statistical significance)
• ignoring secondary metrics that might reveal negative impacts
• not establishing clear baseline measurements before testing
• running experiments without sufficient traffic for meaningful results
• assuming correlation equals causation in results analysis