Texas State Historical Association

How an alternate offer, given at the right time, affected emails acquired

Experiment ID: #11687

Texas State Historical Association

Experiment Summary

Timeframe: 10/12/2015 - 10/19/2015

The Texas State Historical Association had been running an email acquisition campaign around their latest eBook, The Civil War in the Lone Star State. Visitors would be given the eBook in exchange for their email address. This tactic had proven successful with previous eBooks and had historically received a 7% to 10% email acquisition rate. While this is a decent conversion rate, we wanted to find a way to improve this.

We had the hypothesis that we may be able to acquire additional email addresses if we could give the visitors an alternative offer. We didn’t want to compete with the primary offer of the eBook so we decided to only show the alternative offer if the visitors showed signs that they were leaving the page.  We created a treatment that would show a popup offering a free subscription to their Texas Day-by-Day emails when visitors displayed an “exit intent” by moving their mouse toward the back button.

Research Question

Will an alternate offer, shown when the visitors are leaving the name acquisition page, increase the number of emails acquired?

Design

C: eBook Only
T1: Exit Intent Offer

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: eBook Only 7.4%
T1: Exit Intent Offer 10.2%36.9% 99.9%

This experiment has a required sample size of 911 in order to be valid. Since the experiment had a total sample size of 10,596, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 36.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

This alternative offer, shown when a visitor was leaving the page, ended up increasing the number of emails acquired by 36.9%.  What is interesting about these results is that this lift was not achieved just by the offer on the popup. It would seem that, when the popup was shown to people, many visitors would go to close the window and then ended up downloading the ebook after all.  This popup served to arrest the attention of the visitor right before they committed to the decision to leave the page.


Experiment Documented by NextAfter

Question about experiment #11687

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.