How a Topical Inline Email Offer Impacts Email Acquisition Rate Experiment ID: #6897

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 5/30/2017 - 6/30/2017

Alliance Defending Freedom has identified their blog as a high-traffic section of their site. They noticed, however, that too many mobile visitors were leaving without signing up to receive more Alliance Defending Freedom content. To assist in email acquisition, they decided to add an inline offer for mobile visitors that prompted them to ‘stay informed to changes to your religious freedom,’ by signing up to receive the latest generic updates from Alliance Defending Freedom. After noticing that this inline offer increased email acquisition, they decided to create more specific inline offers for each major topic: Sanctity of life, religious liberty, and marriage and family. They set these treatments up against the more generic control, and documented the results.

Research Question

Will showing someone an email acquisition offer that is specific to the article they are on increase the likelihood of conversion?


C: General Inline Email Offer
T1: Topical Inline Email Offer


Treatment Name Conv. Rate Relative Difference Confidence
C: General Inline Email Offer 0.29%
T1: Topical Inline Email Offer 0.24% -17.2% 71.6%

This experiment has a required sample size of 83,225 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

The results of this test did not validate. However, we can still draw some conclusions from the results. The addition of the specific inline offer did not dramatically increase or decrease email acquisition for mobile visitors. This may indicate that blog visitors were not drawn to one particularly topic but rather Alliance Defending Freedom’s high level value proposition. It may also indicate that the more generic inline offer had a great perceived value (i.e. updates on sanctity of life vs. updates on sanctity of life, religious liberty, and marriage and family). Though not statistically valid, this test gave Alliance Defending Freedom a head start in understanding the motivations and behavior of mobile visitors.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.

Pick your format of choice:

Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.