Why low-conversion businesses need exponentially more traffic
Sample size grows nonlinearly with three variables: baseline conversion rate, minimum detectable effect (MDE), and statistical power. A test on a metric with 50 percent baseline conversion and 5 percent MDE needs about 1,000 visitors. The same test on a metric with 5 percent baseline and 5 percent MDE (absolute change, not percentage) needs 25,000 visitors. The same test with a 2 percent MDE needs 156,000 visitors. This exponential relationship explains why testing rare events (like B2B enterprise closes) becomes mathematically impractical.
The tradeoff is whether to test what you can afford to measure (larger MDE = smaller sample = faster tests) or what's truly meaningful to your business (smaller MDE = larger sample = longer tests). Many teams resolve this by testing at different granularities: testing headlines weekly (large MDE), testing final conversion monthly (smaller MDE).
Practical constraints on sample size and speed
A business with 1,000 monthly transactions testing a metric with 1 percent baseline and 0.5 percent absolute MDE needs 62,500 samples. At 1,000 transactions per month, that's six months of data. At that timeline, you'll run two tests per year on that metric, limiting learning velocity. The solution is either increase traffic volume, accept larger MDE (measuring only 1 percent improvements), or focus on higher-baseline metrics where sample sizes are tractable. Fast-moving businesses often test on scaled-up metrics (revenue per user instead of conversion rate) to compress sample sizes.