What impact does a matching gift have on donor conversion?
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 10/01/2018 - 10/11/2018
Alliance Defending Freedom had a unique opportunity during their most recent high urgency campaign. They had secured a matching gift and the campaign they were running was being executed solely through email. This meant they could run an experiment to validate the impact of a matching gift without the risk of cross contaminating the control segment. (This can often happen if there are banner ads in addition to the email program which may cause subscribers in the control group to see messaging intended only for the treatment.)
They decided to split their subscriber file (a random split for both donors and nondonors) and maintain this split for the initial messaging of the campaign. They then sent the control group campaign messaging but without the mention of a matching gift. For the treatment group, they used the same basic messaging but incorporated language about doubling their gift and a deadline to take advantage of the match. They wanted to understand the impact this tactic would have on overall conversion.
Research Question
Will including a matching gift have a positive impact on donor conversion.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Matching Gift | 0.21% | ||
| T1: | Matching Gift | 0.39% | 87.9% | 100.0% |
This experiment has a required sample size of 6,896 in order to be valid. Since the experiment had a total sample size of 138,282, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 87.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
The images above are the first email in the campaign; this same style of messaging was used in the first three emails of the campaign. After those three emails, we saw a 87.9% increase in donor conversion for the subscribers that were a part of the matching gift group. This is a great reminder of the perceived impact this kind of offer can have for potential donor conversion.