How the addition of a reader’s review for a giving premium impacts donor conversion
Harvest Ministries
Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.
Experiment Summary
Timeframe: 02/14/2017 - 02/21/2017
Every month Harvest Ministries sends an appeal that offers a premium to people when they give a donation. The primary value proposition on the donation page is about the premium with the secondary message focused on how a person’s donation will help Harvest Ministries in their mission. They hypothesized if adding a reader’s review of the premium offer to the donation page would help strengthen the primary value proposition and increase donations. They developed a treatment that featured a reader’s review while all other elements on the page remained the same as the control. They A/B split their traffic to test it.
Research Question
Would a reader’s review of the giving premium help increase donations?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 7.8% | ||
| T1: | With Review | 9.9% | 26.6% | 95.5% |
This experiment has a required sample size of 1,444 in order to be valid. Since the experiment had a total sample size of 2,984, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 26.6% increase in conversion rate
× 0% increase in average gift
Key Learnings
The addition of a reader’s review for the premium on the donation page helped increase donations for their monthly appeal by 26%! What we can learn from this is that the review:
- Strengthened the force of the value proposition on the page
- Added credibility and relevance to the premium offer by giving a real-life testimony of the value it added to a person’s life
Because the monthly appeals are heavily focused on the giving premium, we will continue to test and optimize ways to reinforce the premium’s value proposition in both the email and donation page copy.