How removing the recommended donation amount increased the perceived value of an offer
Every Home for Christ
Experiment Summary
Timeframe: 03/20/2017 - 04/07/2017
Every Home for Christ was offering their book Look What God is Doing as a free offer for donors and potential donors to develop a deeper understanding of the impact that their organization was having across the globe.
After someone requested the book, they were shown a Thank You page with a with a follow-up donation ask. In the control version of this page, Every Home for Christ was recommending a $10 donation. In exchange for this donation, they would send the donor a free copy of another book titled The Hour That Changes the World.
They had the hypothesis that this recommended giving level could be keeping donors from giving at their full capacity. So they created two treatments. The first treatment recommended a $25 donation. The second treatment did not recommend an amount, but left an open field for the donor to give a gift of any amount.
Research Question
Will removing the recommended gift amount affect donations?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | $10 Offer | 2.2% | ||
| T1: | $25 Offer | 1.3% | -42.0% | 75.8% |
| T2: | Any $ Offer | 8.2% | 276.5% | 100.0% |
This experiment has a required sample size of 90 in order to be valid. Since the experiment had a total sample size of 1,675, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 276.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
Treatment 2, the version without a donation recommendation, saw a 276% increase in donations. This tells us that the initial recommended gift amount may have been holding their donors and potential donors back from giving at their full potential.
By removing the recommendation, the value of the offer was increased in the mind of the donor, leading to a higher conversion rate.
From here, Every Home for Christ will be monitoring the rate at which these donors give a 2nd gift in order to determine if this winning treatment has any effect on the longevity of the donor.