How the content on the exit-intent offer is influenced by the content of the page Experiment ID: #8022
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Timeframe: 9/9/2017 - 12/5/2017
As part of their campaign to educate the general public on the Jack Phillips case, Alliance Defending Freedom created a landing page that would serve as both a means for education and email acquisition. The primary call-to-action was an email acquisition offer to ‘stay up-to-date on Jack Phillips.’ To take advantage of all of the traffic going to the page, we wanted to allow implement an exit-intent offer to potentially capture people prior to them leaving the site.
In similar tests in the past, we tested offers specific to the content on the page. In these situations, the more general ADF value proposition had won out every time. However, in this situation, we hypothesized that the relevance of Jack Phillips may make this situation different. We decided to test the two treatments as a result.
Will an exit intent offer that is specific to the content on the page (Jack Phillips) increase the email acquisition rate?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|T1:||Jack Specific Offer||2.3%||51.5%||99.0%|
This experiment has a required sample size of 2,399 in order to be valid. Since the experiment had a total sample size of 8,354, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 51.5% increase in conversion rate
× 0% increase in average gift
In this situation, we found that the Jack Phillips’ specific offer increased email acquisition by 51%. It turns out that in this specific situation, the relevance of Jack Phillips increased the appeal of the “Stay Informed” offer more than the general ADF offer. In future campaigns, this is something we will want to evaluate on launch as it tends to vary from client to client.