How an alternate offer, given at the right time, affected emails acquired
Texas State Historical Association
Experiment Summary
Timeframe: 10/12/2015 - 10/19/2015
The Texas State Historical Association had been running an email acquisition campaign around their latest eBook, The Civil War in the Lone Star State. Visitors would be given the eBook in exchange for their email address. This tactic had proven successful with previous eBooks and had historically received a 7% to 10% email acquisition rate. While this is a decent conversion rate, we wanted to find a way to improve this.
We had the hypothesis that we may be able to acquire additional email addresses if we could give the visitors an alternative offer. We didn’t want to compete with the primary offer of the eBook so we decided to only show the alternative offer if the visitors showed signs that they were leaving the page. We created a treatment that would show a popup offering a free subscription to their Texas Day-by-Day emails when visitors displayed an “exit intent” by moving their mouse toward the back button.
Research Question
Will an alternate offer, shown when the visitors are leaving the name acquisition page, increase the number of emails acquired?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | eBook Only | 7.4% | ||
| T1: | Exit Intent Offer | 10.2% | 36.9% | 99.9% |
This experiment has a required sample size of 911 in order to be valid. Since the experiment had a total sample size of 10,596, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 36.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
This alternative offer, shown when a visitor was leaving the page, ended up increasing the number of emails acquired by 36.9%. What is interesting about these results is that this lift was not achieved just by the offer on the popup. It would seem that, when the popup was shown to people, many visitors would go to close the window and then ended up downloading the ebook after all. This popup served to arrest the attention of the visitor right before they committed to the decision to leave the page.