How the content on the exit-intent offer is influenced by the content of the page Experiment ID: #8022

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 9/9/2017 - 12/5/2017

As part of their campaign to educate the general public on the Jack Phillips case, Alliance Defending Freedom created a landing page that would serve as both a means for education and email acquisition. The primary call-to-action was an email acquisition offer to ‘stay up-to-date on Jack Phillips.’ To take advantage of all of the traffic going to the page, we wanted to allow implement an exit-intent offer to potentially capture people prior to them leaving the site.

In similar tests in the past, we tested offers specific to the content on the page. In these situations, the more general ADF value proposition had won out every time. However, in this situation, we hypothesized that the relevance of Jack Phillips may make this situation different. We decided to test the two treatments as a result.

Research Question

Will an exit intent offer that is specific to the content on the page (Jack Phillips) increase the email acquisition rate?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: ADF General
T1: Jack Specific Offer

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: ADF General 1.5%
T1: Jack Specific Offer 2.3% 51.5% 99.0%

This experiment has a required sample size of 2,399 in order to be valid. Since the experiment had a total sample size of 8,354, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 51.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

In this situation, we found that the Jack Phillips’ specific offer increased email acquisition by 51%. It turns out that in this specific situation, the relevance of Jack Phillips increased the appeal of the “Stay Informed” offer more than the general ADF offer. In future campaigns, this is something we will want to evaluate on launch as it tends to vary from client to client.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.