Harvest Ministries

How an additional verification screen in the donation process affects donor conversion

Experiment ID: #3711

Harvest Ministries

Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.

Experiment Summary

Timeframe: 04/01/2016 - 04/18/2016

Harvest Ministries’ online donation process had a verification screen before the donation was finalized. This sort of “last step” is common in the eCommerce world to reduce the number of accidental transactions. However, in Harvest’s donation process, the language on the page was unclear and the call-to-action was not clearly prioritized. This made us curious if there was a segment of people who thought the donation process was complete and whose gift was never processed.

We hypothesized that removing this step would reveal whether this was, in fact, true. The Harvest team set up an alternate donation process that omitted this step and we ran an A/B test to determine a result.

Research Question

Will removal of the verification screen decrease friction and result in more completed donations?

Design

C: With Verification Page
T1: Thank-you Page

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: With Verification Page 12.4%
T1: Thank-you Page 27.4%121.5% 100.0%

This experiment has a required sample size of 63 in order to be valid. Since the experiment had a total sample size of 1,421, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 121.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

Removing the extra verification screen increased donations by 121%! We hypothesized that the additional screen was doing one of two things:

  1. Convincing donors that the process was done, when their gift was, in fact, not confirmed
  2. Adding an extra layer of friction to the process that caused potential donors to bail out of the process.

The removal of this step showed that many more gifts were intended to be made than were actually captured. This is a fascinating learning as we examine the steps of donation processes from many organizations. Sometimes friction is helpful, but sometimes it can be turning donors away.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #3711

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.