How additional friction from a verification screen affects revenue Experiment ID: #3712

Harvest Ministries

Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.

Experiment Summary

Timeframe: 4/8/2016 - 4/18/2016

Harvest Ministries’ online donation process had a verification screen before the donation was finalized. This sort of “last step” is common in the eCommerce world to reduce the number of accidental transactions. However, in Harvest’s donation process, the language on the page was unclear and the call-to-action was not clearly prioritized. This made us curious if there was a segment of people who thought the donation process was complete and whose gift was never processed.

We hypothesized that removing this step would reveal whether this was, in fact, true. The Harvest team set up an alternate donation process that omitted this step and we ran an A/B test to determine a result.

Research Question

Will removal of the verification screen decrease friction and result in more donation revenue?

Design

C: With Verification Screen
T1: Thank-You Page

Results

Treatment Name Revenue per Visitor Relative Difference Confidence Average Gift
C: With Verification Screen $12.66 $102.50
T1: Thank-You Page $34.90 175.6% 100.0% $122.54

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 130.6% increase in conversion rate
× 19.5% increase in average gift
175.6% increase in revenue

Key Learnings

The treatment donation flow resulted in 175% more revenue than the control. Through a combination of increased conversion and slightly increased average gift (which was most likely due to a larger sample size of transactions), removing the verification screen resulted in nearly three times the revenue.

While the experiment produced a 121% increase in conversion,  we must also use revenue as a metric to make sure that increased conversions don’t result in a reduced average gift, which could actually bring total revenue down.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.