How the presentation of the offer impacts email acquisition
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 03/13/2018 - 03/20/2018
Roughly two years ago, Alliance Defending Freedom went through a series of experiments to discover the optimal email acquisition tactic for their desktop blog traffic. After testing slide outs, embedded forms, and exit intents, we found that the exit intent was the ideal acquisition method. However, after using this for two years, we wanted to revalidate this method to ensure nothing had changed.
Using the most recent email acquisition offer, a statement of belief surrounding the NIFLA v. Becerra case, we wanted to see if an inline embedded form would be more successful than the exit intent. One caveat, we were unable to embed the entire Statement of Belief on the exit intent due to the length of copy so it had to link to a landing page with additional copy on it.
Research Question
Will an embedded statement of belief increase email acquisition over the standard exit intent?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Exit Intent | 0.72% | ||
| T1: | Inline Offer | 6.6% | 812.1% | 100.0% |
This experiment has a required sample size of 79 in order to be valid. Since the experiment had a total sample size of 5,819, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 812.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
The embedded statement of belief increased email acquisition by 812%. There are two important factors to point out with this experiment:
- We significantly lengthened the amount of copy on the blog by embedding the full statement of belief on the blog
- The statement of belief is only shown to people on the first page they try to leave on. If they close the window but don’t leave the site, it will hide it on future pages
This shows the power of exposing the visitors to the offer multiple times (by keeping it on every page) and how the multi-step funnel allows for significant fall off. For future tests, we will want to continue to validate not only this tactic but also the offer itself. Does including an offer that is more closely tied to the content of the article help increase email acquisition?