Alliance Defending Freedom

What impact does a matching gift have on donor conversion?

Experiment ID: #7367

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 10/01/2018 - 10/11/2018

Alliance Defending Freedom had a unique opportunity during their most recent high urgency campaign. They had secured a matching gift and the campaign they were running was being executed solely through email. This meant they could run an experiment to validate the impact of a matching gift without the risk of cross contaminating the control segment. (This can often happen if there are banner ads in addition to the email program which may cause subscribers in the control group to see messaging intended only for the treatment.)

They decided to split their subscriber file (a random split for both donors and nondonors) and maintain this split for the initial messaging of the campaign. They then sent the control group campaign messaging but without the mention of a matching gift. For the treatment group, they used the same basic messaging but incorporated language about doubling their gift and a deadline to take advantage of the match. They wanted to understand the impact this tactic would have on overall conversion.

Research Question

Will including a matching gift have a positive impact on donor conversion.

Design

C: No Matching Gift
T1: Matching Gift

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: No Matching Gift 0.21%
T1: Matching Gift 0.39%87.9% 100.0%

This experiment has a required sample size of 6,896 in order to be valid. Since the experiment had a total sample size of 138,282, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 87.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

The images above are the first email in the campaign; this same style of messaging was used in the first three emails of the campaign. After those three emails, we saw a 87.9% increase in donor conversion for the subscribers that were a part of the matching gift group. This is a great reminder of the perceived impact this kind of offer can have for potential donor conversion.


Experiment Documented by NextAfter

Question about experiment #7367

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.