How asking lapsed donors to be one of a few donors to give a specific amount impacted donor conversion rate during a matching gift campaign Experiment ID: #22982

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Ended On: 8/4/2020

As a part of The Heritage Foundation’s matching gift campaign (called “The Board Challenge”), we wanted to experiment with the ask amount and approach. Typically, the organization has used “make your best gift today” kind of language within their control copy.

In an attempt to set a smaller, more reasonable target, we wanted to experiment with driving lapsed donors to be “one of just X donors we need today to make a gift of Y.

The idea behind this is to (a) set a relevant ask amount for the donors, but also (b) allow them to see that they can help achieve a “small quantity” goal at that amount, as well.

Research Question

How does communicating a specific quantity of donors we’re asking to make a gift relevant to the lapsed donor impact donor conversion or revenue?

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: "Best Gift" 0.22%
T1: "One of a few" 0.24% 10.8% 58.9%

This experiment has a required sample size of 321,418 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

We did not validate an increase, we did see we did observe a +10.8% increase in lapsed donor conversion rate — albeit at only a 58% level of confidence.

Further, the revenue was relatively flat between the two segments (only showing a +7.2% increase, with a level of confidence of 60.9%). It was promising, however, that we also observed that 29% of the people in the treatment segment decided to give more than the amount that was asked for, while another 41% gave the exact amount that was asked of them.

Conclusion: Perhaps further experimentation with the ask amounts is advisable for lapsed donors, but this directionally showed a lift in revenue.


Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.