How finding the right incentive affects the email capture rate Experiment ID: #1355

Texas State Historical Association

Experiment Summary

Timeframe: 3/24/2014 - 7/4/2014

The Texas State Historical Association (TSHA) has a large volume of traffic to their websites each month. In order to capitalize on this traffic, they have set up several email capture offers to use at different times of the year.  One of those offers was a 10 question quiz named Are You Smarter Than a Texas 7th Grader. At the end of the quiz, the visitors are offered a digital resource in exchange for their email address.

We wanted to find the digital resource that would be most appealing to the TSHA audience so we created an experiment that would allow us to test out several offers.  We offered visitors either a section of the TSHA’s Texas Almanac, a free eBook entitled the Battle of the Alamo, or a collection of popular articles from their publication Southwestern Historical Quarterly.

Research Question

Which incentive is the most effective in capturing an email address?

Design

C: Texas Almanac
T1: Alamo eBook
T2: Southwestern Historical Quarterly

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Texas Almanac 13.0%
T1: Alamo eBook 19.5% 50.2% 100.0%
T2: Southwestern Historical Quarterly 14.5% 11.5% 99.4%

This experiment has a required sample size of 379 in order to be valid. Since the experiment had a total sample size of 35,536, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 50.2% increase in conversion rate
× 0% increase in average gift

Key Learnings

The Battle of the Alamo eBook proved to be the most popular offer when it lifted email capture rate by 50%. This same fact held true a year later when we tested this same eBook against the full eBook of the Texas Almanac. However, a year later the impact was not as significant when the full Texas Almanac was offered.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.





Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.


Pick your format of choice:

Related Experiments

This experiment is part of a series of experiments aimed at improving overall results. Take a look at some of the other iterations:


Experiment Documented by...

Tim Kachuriak

Tim is the Chief Innovation & Optimization Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.