Texas State Historical Association

How the right premium affects conversion

Experiment ID: #2347

Texas State Historical Association

Experiment Summary

Timeframe: 10/15/2015 - 10/26/2015

At the Texas State Historical Association, our previous tests had proven how popular the Texas Almanac was for their audience. In many campaigns, we had started using it as a premium for members who joined at the $50 or more level.

While looking through their website for optimization opportunities, we found that the primary donation page had not been modified for a while. We decided to use this page to create a test that would allow us to isolate the impact the Almanac has on overall donor conversion.

Research Question

What will be the impact of offering the Texas Almanac as a premium?


C: No Almanac
T1: Almanac


 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: No Almanac 2.2%$50.00
T1: Almanac 5.1%135.4% 92.0%$55.77

This experiment has a required sample size of 317 in order to be valid. Since the experiment had a total sample size of 484, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

By offering the Texas Almanac as the primary incentive on the page, we were able to increase the member acquisition by 135%.

Every premium has an associated cost that needs to be accounted for when determining its overall effectiveness.  In this situation, a 135% increase to the number of members acquired with an 11% increase to average gift is more than worth the cost associated with sending out the Almanac to each new member.

Experiment Documented by NextAfter

Question about experiment #2347

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.