Which email sign up headline produces the highest conversion rate

Experiment ID: #11716


Experiment Summary

Timeframe: 08/12/2015 - 08/28/2015

Every once in a while we get around to running tests on our own website to make sure NextAfter is practicing what we preach. So we set up two different email signup pages encouraging visitors to get the This vs. That weekly email which showcases one of our latest test results. The goal was to see which email sign up value proposition would drive the most conversions.

Research Question

What email value proposition will produce the most email sign ups?


C: Test your marketing intuition
T1: Get sent our latest test


  Treatment Name Conv. Rate Relative Difference Confidence
C: Test your marketing intuition 1.6%
T1: Get sent our latest test 9.8% 506.5% 98.1%

This experiment has a required sample size of 70 in order to be valid. Since the experiment had a total sample size of 154, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 506.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

This test is another example of how clarity trumps persuasion. Clearly stating what a person is signing up for, getting a test each week, is a better description than talking about how a person will benefit. The surprising aspect to us was that even with the low traffic to the page the test validated.

The copy used to promote offers is the most important element in getting someone to say yes. Just throwing something up on the page and not testing it would have seriously affected the number of email sign-ups the page generated.

Experiment Documented by Dan VanMilligan
Dan VanMilligan is Implementation and Optimization Specialist at NextAfter.

Question about experiment #11716

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.