How exit intent offers can help you acquire more email addresses Experiment ID: #6029

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 1/16/2017 - 1/27/2017

The Alliance Defending Freedom blog gets high volumes of traffic with the majority coming from social media. Over a series of past experiments, we had determined a strong message that maximized the number of emails acquired using a slide out offer that was visible when visitors scrolled down the page.

We had the hypothesis that a new delivery mechanism for the value proposition could be more effective. We decided to test out an exit-intent popup that would appear when visitors moved their mouse outside of the browser window. We made sure to target this experiment at desktop users only as mobile users would not be able to experience the effect.

Research Question

Will an exit intent or slide out offer have the best email acquisition rate?


C: Slide Out
T1: Exit Intent Popup


Treatment Name Conv. Rate Relative Difference Confidence
C: Slide Out 0.24%
T1: Exit Intent Popup 0.79% 228.8% 100.0%

This experiment has a required sample size of 1,290 in order to be valid. Since the experiment had a total sample size of 19,740, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 228.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

The exit-intent offer significantly increased the email acquisition rate. It is our hypothesis that the exit intent overcame three issues that the slide out offer presented:

  1. The slide out offer was only seen by visitors that made it 50% of the way down the page. This means a significant portion of the audience would never see it.
  2. The slide out offer disrupted the reading process. The goal of these offers is to create a disruption to arrest visitor attention but by using an exit intent popup, we could time that disruption once the visitor was done reading the article.
  3. It was a completely new tactic. The slide out offer had been running for over a year. Simply adding a new way to deliver the email offer may have grabbed visitor attention. For this reason, we will continue to monitor the acquisition rate to ensure it does not drop over time.

Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.