How finding the right incentive affects the email capture rate
Texas State Historical Association
Experiment Summary
Timeframe: 03/24/2014 - 07/04/2014
The Texas State Historical Association (TSHA) has a large volume of traffic to their websites each month. In order to capitalize on this traffic, they have set up several email capture offers to use at different times of the year. One of those offers was a 10 question quiz named Are You Smarter Than a Texas 7th Grader. At the end of the quiz, the visitors are offered a digital resource in exchange for their email address.
We wanted to find the digital resource that would be most appealing to the TSHA audience so we created an experiment that would allow us to test out several offers. We offered visitors either a section of the TSHA’s Texas Almanac, a free eBook entitled the Battle of the Alamo, or a collection of popular articles from their publication Southwestern Historical Quarterly.
Research Question
Which incentive is the most effective in capturing an email address?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Texas Almanac | 13.0% | ||
| T1: | Alamo eBook | 19.5% | 50.2% | 100.0% |
| T2: | Southwestern Historical Quarterly | 14.5% | 11.5% | 99.4% |
This experiment has a required sample size of 379 in order to be valid. Since the experiment had a total sample size of 35,536, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 50.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
The Battle of the Alamo eBook proved to be the most popular offer when it lifted email capture rate by 50%. This same fact held true a year later when we tested this same eBook against the full eBook of the Texas Almanac. However, a year later the impact was not as significant when the full Texas Almanac was offered.