How a radical redesign of the main donation form impacted giving Experiment ID: #17692

Buckner International

Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.

Experiment Summary

Timeframe: 6/21/2019 - 8/15/2019

Given the volume of traffic and the average gift size, we have struggled to validate an experiment on Buckner’s main donation form in the past. With this in mind, we decided to test out a radical new donation form that changed multiple elements that we’ve seen work in the past with similar organizations.

Specifically, we made the following changes:

  • We modified the form headers to be numbers steps instead of just labels.
  • We removed the gift array which we’ve seen help with organizations that have a large volume of mid-level and major donors
  • We increased the perceived security near the credit card form
  • We reiterated the gift size and impact near the donation button

Research Question

Will a radical redesign of the donation form increase overall donor conversion?

Design

C: Control
T1: With Updates

Results

Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 18.8% $0.00
T1: With Updates 11.1% -41.1% 99.6% $0.00

This experiment has a required sample size of 160 in order to be valid. Since the experiment had a total sample size of 707, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 41.1% decrease in conversion rate
× 0% increase in average gift

Key Learnings

After running for nearly two months, we observed a 41% decrease in overall donor conversion. When we parsed through the data, what we discovered is that the donor decrease was only seen in the segment of donors giving less than $100. The number of donors giving above that level actually increased in volume.

We were also concurrently running a similar experiment on the Shoes for Orphan Souls (SOS) donation page and it observed an increase in giving using similar changes. However, the SOS donation page had a targeted ask for $25 right before the gift amount section.

Based upon these two factors, it is our belief that removing the gift array buttons negatively impacted the small dollar donor conversion since we did not properly frame up the ideal gift size for the audience. Even though the SOS page had the donation buttons removed, it had the targeted ask which effectively replaced the impact of the buttons. It is our recommendation to relaunch this experiment but add the donation buttons back in to validate this hypothesis.


Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.