Texas State Historical Association

How finding the right incentive affects the email capture rate

Experiment ID: #11709

Texas State Historical Association

Experiment Summary

Timeframe: 03/24/2014 - 07/04/2014

The Texas State Historical Association (TSHA) has a large volume of traffic to their websites each month. In order to capitalize on this traffic, they have set up several email capture offers to use at different times of the year.  One of those offers was a 10 question quiz named Are You Smarter Than a Texas 7th Grader. At the end of the quiz, the visitors are offered a digital resource in exchange for their email address.

We wanted to find the digital resource that would be most appealing to the TSHA audience so we created an experiment that would allow us to test out several offers.  We offered visitors either a section of the TSHA’s Texas Almanac, a free eBook entitled the Battle of the Alamo, or a collection of popular articles from their publication Southwestern Historical Quarterly.

Research Question

Which incentive is the most effective in capturing an email address?

Design

C: Texas Almanac
T1: Alamo eBook
T2: Southwestern Historical Quarterly

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Texas Almanac 13.0%
T1: Alamo eBook 19.5%50.2% 100.0%
T2: Southwestern Historical Quarterly 14.5%11.5% 99.4%

This experiment has a required sample size of 379 in order to be valid. Since the experiment had a total sample size of 35,536, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 50.2% increase in conversion rate
× 0% increase in average gift

Key Learnings

The Battle of the Alamo eBook proved to be the most popular offer when it lifted email capture rate by 50%. This same fact held true a year later when we tested this same eBook against the full eBook of the Texas Almanac. However, a year later the impact was not as significant when the full Texas Almanac was offered.


Experiment Documented by Tim Kachuriak
Tim Kachuriak is Chief Innovation and Optimization Officer of NextAfter.

Question about experiment #11709

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.