How testing into a new offer affects name acquisition
Texas State Historical Association
Experiment Summary
Timeframe: 06/01/2015 - 06/05/2015
To take advantage of their high website traffic, the Texas State Historical Association has been offering a free eBook version of their popular 2014-2015 Texas Almanac in exchange for visitor email addresses. This name acquisition offer has been running for the past 6 months and continues to be a high performer. However, with the release of a recent History Channel show, the motivation of website visitors had shifted and we wanted to see if a different offer could further improve the acquisition efforts.
As a result, the TSHA ran a three way split on the traffic offering three different eBook options, the Texas Almanac, The Battle of the Alamo, and Remember Goliad! We wanted to see which of these offers could collect the most new email addresses for the organization.
Research Question
Which name acquisition offer produces the most new email addresses for the file?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | 2014-15 Texas Almanac eBook | 1.3% | ||
T1: | The Battle of the Alamo eBook | 1.6% | 23.6% | 99.7% |
T2: | Remember Goliad! eBook | 0.99% | -21.1% | 99.7% |
This experiment has a required sample size of 4,564 in order to be valid. Since the experiment had a total sample size of 85,932, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 23.6% increase in conversion rate
× 0% increase in average gift
Key Learnings
The Alamo eBook improved email acquisition by 23.6% while maintaining the immediate donor conversion on the follow up screen. This increased volume of email addresses will allow for additional conversion opportunities down the line.
As a result, the TSHA will be changing offers to just focus on the Alamo eBook for the next several weeks. Once the show is over, we will want to re-validate these results to ensure the motivation (and corresponding conversion) does not change.
Question about experiment #11702
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.