How the style of offer impact email acquisition Experiment ID: #12633

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 2/27/2018 - 3/13/2018

In a previous experiment for Alliance Defending Freedom, we had found that a Statement of Belief dramatically outperformed the standard email newsletter offer for visitors coming from paid social media ads. Based upon these results, we wanted to see if the same impact would be seen if we applied the Statement of Belief offer to organic visitors to the blog. The ADF blog is a significant source of visitors to the site and, over the years, has become a great source for email acquisition.

Research Question

Will a statement of belief offer increase email acquisition compared to the generic email newsletter offer?


C: Email Newsletter
T1: Statement of Belief


Treatment Name Conv. Rate Relative Difference Confidence
C: Email Newsletter 0.62%
T1: Statement of Belief 1.2% 98.0% 100.0%

This experiment has a required sample size of 1,924 in order to be valid. Since the experiment had a total sample size of 30,772, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 98.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

Through this experiment, we were able to increase email acquisition by 98% for both mobile and desktop visitors. This is a significant leap forward in email acquisition for the visitors being driven to the blog. It is also a great indication that we need to expand our thinking when it comes to the offers we’re presenting to visitors. We need to make sure they are unique and appealing to our ideal audience; we will also need to make sure these offers are constantly being tested to stay relevant with the audience.

Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.