How the presentation of the offer impacts email acquisition Experiment ID: #8749

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 3/13/2018 - 3/20/2018

Roughly two years ago, Alliance Defending Freedom went through a series of experiments to discover the optimal email acquisition tactic for their desktop blog traffic. After testing slide outs, embedded forms, and exit intents, we found that the exit intent was the ideal acquisition method. However, after using this for two years, we wanted to revalidate this method to ensure nothing had changed.

Using the most recent email acquisition offer, a statement of belief surrounding the NIFLA v. Becerra case, we wanted to see if an inline embedded form would be more successful than the exit intent. One caveat, we were unable to embed the entire Statement of Belief on the exit intent due to the length of copy so it had to link to a landing page with additional copy on it.

Research Question

Will an embedded statement of belief increase email acquisition over the standard exit intent?

Design

C: SOB Button
T1: SOB Inline

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: SOB Button 0.72%
T1: SOB Inline 6.6% 812.1% 100.0%

This experiment has a required sample size of 79 in order to be valid. Since the experiment had a total sample size of 5,819, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 812.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

The embedded statement of belief increased email acquisition by 812%. There are two important factors to point out with this experiment:

  • We significantly lengthened the amount of copy on the blog by embedding the full statement of belief on the blog
  • The statement of belief is only shown to people on the first page they try to leave on. If they close the window but don’t leave the site, it will hide it on future pages

This shows the power of exposing the visitors to the offer multiple times (by keeping it on every page) and how the multi-step funnel allows for significant fall off. For future tests, we will want to continue to validate not only this tactic but also the offer itself. Does including an offer that is more closely tied to the content of the article help increase email acquisition?


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.