How a matching gift influences donor conversion
Museum of the Bible
Experiment Summary
Timeframe: 12/30/2018 - 01/01/2019
During the final days of their year end campaign, Museum of the Bible sent an email to their house file promoting the December 31 deadline and $1 million dollar goal. In addition to these two factors, the museum had also secured a matching gift that would double every donation received. It was decided to test out the impact of this matching gift via email. To do this, two similar but different emails were crafted with the primary change being the mention of the matching gift. Both emails were directed to the same donation page where the matching gift was not included in the copy.
Research Question
Will promoting a matching gift opportunity improve donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Match | 0.10% | ||
| T1: | With Match | 0.16% | 50.3% | 95.7% |
This experiment has a required sample size of 36,421 in order to be valid. Since the experiment had a total sample size of 77,155, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 50.3% increase in conversion rate
× 0% increase in average gift
Key Learnings
With a 50% increase to donations and a statistical level of confidence of 95%, we can be certain that incorporating the matching gift into the email increased donor acquisition. It is possible this lift could have been increased even further if we were to carry this concept through to the donation page. During this campaign, the emails all used the same, generic donation page. Our hypothesis is that if we reinforced the matching gift on the page too, we could potentially move even more potential donors to convert.
As a note, since the difference between the two emails was in the content itself, the sample size is based upon the number of subscribers that opened the email.