Heritage Action for America

How value proposition impacts donor conversion for a serial ask

Experiment ID: #2472

Heritage Action for America

Experiment Summary

Timeframe: 12/21/2017 - 12/26/2017

For Heritage Action, the serial ask shown to The Heritage Foundation online donors has become a significant source of revenue. It works by allowing Heritage donors to complete their gift and then immediately positioning a second ask for Heritage Action.

The copy that is commonly used in the serial ask has been in place since it was first launched. There have been a handful of attempts to test this language but without any valid results.

With the high motivation of year end giving, we knew that we would be getting significantly more donors than normal. Given this increased volume, we wanted to try a new value proposition that aligned better with the current political climate.

Research Question

Will the new Heritage Action value proposition increase overall conversion?

Design

C: Original
T1: New HAFA Value Prop

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Original 11.2%
T1: New HAFA Value Prop 25.2%124.8% 99.5%

This experiment has a required sample size of 59 in order to be valid. Since the experiment had a total sample size of 226, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 124.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

After running for 5 days, we saw a 125% lift in conversion to Heritage Action donors. Given the importance of year end, we decided to roll this new treatment for the rest of the year. However, this is something that we will want to revisit in the following month. We want to be sure it has the same impact when donor motivations are lower.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #2472

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.