How value proposition impacts donor conversion for a serial ask
Heritage Action for America
Experiment Summary
Timeframe: 12/21/2017 - 12/26/2017
For Heritage Action, the serial ask shown to The Heritage Foundation online donors has become a significant source of revenue. It works by allowing Heritage donors to complete their gift and then immediately positioning a second ask for Heritage Action.
The copy that is commonly used in the serial ask has been in place since it was first launched. There have been a handful of attempts to test this language but without any valid results.
With the high motivation of year end giving, we knew that we would be getting significantly more donors than normal. Given this increased volume, we wanted to try a new value proposition that aligned better with the current political climate.
Research Question
Will the new Heritage Action value proposition increase overall conversion?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Original | 11.2% | ||
T1: | New HAFA Value Prop | 25.2% | 124.8% | 99.5% |
This experiment has a required sample size of 59 in order to be valid. Since the experiment had a total sample size of 226, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 124.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running for 5 days, we saw a 125% lift in conversion to Heritage Action donors. Given the importance of year end, we decided to roll this new treatment for the rest of the year. However, this is something that we will want to revisit in the following month. We want to be sure it has the same impact when donor motivations are lower.
Question about experiment #2472
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.