How a case-specific exit intent offers impacts email acquisition
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 09/09/2017 - 10/17/2017
Alliance Defending Freedom was looking for a new way to improve the exit intent offer that visitors see before leaving their site. They wondered if a more specific call-to-action would cause visitors to take action with more urgency. They decided to test an offer around one of their largest legal cases right now: The defense of Jack Phillips. So they created a treatment version and launched an A/B test to find out.
Research Question
Will an exit intent offer geared towards a particular legal case increase email acquisition?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.6% | ||
| T1: | Treatment 1 | 1.8% | 11.1% | 27.0% |
This experiment has a required sample size of 40,583 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
After running for over a month, this test did not validate for Alliance Defending Freedom. While they did begin to see a lift with the more case-specific messaging, when compared with the mission-specific messaging (the control), the end results were inconclusive. This has prompted further test ideas regarding the nature and offer of exit intent offers.