How a direct ask in an exit intent affects conversion
Buckner International
Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.
Experiment Summary
Timeframe: 05/28/2017 - 06/01/2017
Toward the end of a matching gift campaign, Buckner International wanted to test out an exit intent on their main website promoting the challenge. Since this was the first time this approach had been used, we wanted to find out which approach (a “learn more” or a direct ask) would have the greatest impact on a visitor’s likelihood to make a gift.
Research Question
Will a more direct ask increase traffic or donor conversion rate for an exit intent banner?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Soft CTA | 1.6% | ||
| T1: | Direct Ask CTA | 0.81% | -50.1% | 94.7% |
This experiment has a required sample size of 1,400 in order to be valid. Since the experiment had a total sample size of 2,718, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
We found that the direct ask approach reduced to the number of visitors to the campaign page by 50%. While it did not reach the 95% level of confidence we would strive for, it was quickly approaching validity before the campaign ended. However, the goal was not necessarily to get clicks, it was to get donations.
When we looked at the number of actual donations, there was not a statistically significant difference between the two treatments. We were able to drive more traffic to the page but this did not result in more total donations. For future tests such as this, we will need to go with a more radically different treatments to achieve validity.