The Heritage Foundation

How soliciting from a cultivation personality to a list they are not cultivating impacted donor conversion rate during a matching gift campaign

Experiment ID: #34382

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Ended On: 08/04/2020

The Heritage Foundation’s Director of Membership has been cultivating a portion of the email house file for 9-months now, never having ever solicited the list directly. During this campaign, we wanted to see how the list he was cultivating would respond when he solicited. Furthermore, we also wanted to experiment with how using him as a solicitor would work for the portion of the list he was not cultivating, as well.

Research Question

Will soliciting from the portion of the list not being actively cultivated by the cultivation-only personality increase donor conversion rate?

Design

C: Control
T1: Treatment #1

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.25%
T1: Treatment #1 0.33%30.5% 100.0%

This experiment has a required sample size of 33,745 in order to be valid. Since the experiment had a total sample size of 558,278, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 30.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 99.9% level of confidence, we observed a +30.5% increase in donor conversion rate when using this personality (who had never solicited the audience before) as a solicitor for the list he was not actively cultivating.

We also observed a validated lift in revenue of +25.3% (LoC: 93.4%), as well.

Conclusion: Using a person who has never solicited an audience before within a campaign seems to show an increase in donor conversion rate, considering we also observed a +41% increase in donor conversion rate (99.9% LoC) for this solicitor when asking his cultivation list to give during the campaign, as well. 

In other words, if you have a personality, in a ranking position that has never solicited an audience before, it may make sense to experiment with them as a sender within a campaign, as shown here.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #34382

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.