How removing a challenge grant affects conversion
Media Research Center
The Media Research Center (MRC) is the nation’s premier media watchdog. The sole mission is to expose and neutralize the propaganda arm of the Left: the national news media.
Experiment Summary
Ended On: 04/08/2013
As part of a fundraising campaign, MRC had the opportunity to utilize a challenge grant. When comparing one campaign to the next, we have often seen campaigns with a challenge grant significantly increase both average gift and conversion rate. However, this doesn’t give us an accurate understanding of the exact impact a challenge grant has since there can be many other factors influencing two different campaigns.
As a result, we wanted to create a treatment for a campaign that did not mention the challenge grant to see what kind of impact this would have on giving.
Research Question
How does the presence of a challenge grant affect giving?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Challenge Grant Language | 2.1% | ||
| T1: | No Challenge Grant Language | 1.6% | -24.1% | 97.9% |
This experiment has a required sample size of 5,502 in order to be valid. Since the experiment had a total sample size of 15,149, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 24.1% decrease in conversion rate
× 0% increase in average gift
Key Learnings
The treatment with no challenge grant language saw a 24% decrease in conversion rate. Language around “doubling your gift” appears to provide increased incentive for potential donors to make their gift since they believe that the impact is incrementally more valuable.