How an additional verification screen in the donation process affects donor conversion Experiment ID: #3711
Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.
Timeframe: 4/1/2016 - 4/18/2016
Harvest Ministries’ online donation process had a verification screen before the donation was finalized. This sort of “last step” is common in the eCommerce world to reduce the number of accidental transactions. However, in Harvest’s donation process, the language on the page was unclear and the call-to-action was not clearly prioritized. This made us curious if there was a segment of people who thought the donation process was complete and whose gift was never processed.
We hypothesized that removing this step would reveal whether this was, in fact, true. The Harvest team set up an alternate donation process that omitted this step and we ran an A/B test to determine a result.
Will removal of the verification screen decrease friction and result in more completed donations?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|C:||With Verification Page||12.4%|
This experiment has a required sample size of 63 in order to be valid. Since the experiment had a total sample size of 1,421, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 121.5% increase in conversion rate
× 0% increase in average gift
Removing the extra verification screen increased donations by 121%! We hypothesized that the additional screen was doing one of two things:
- Convincing donors that the process was done, when their gift was, in fact, not confirmed
- Adding an extra layer of friction to the process that caused potential donors to bail out of the process.
The removal of this step showed that many more gifts were intended to be made than were actually captured. This is a fascinating learning as we examine the steps of donation processes from many organizations. Sometimes friction is helpful, but sometimes it can be turning donors away.