How a radical redesign impacted the Shoes for Orphan Souls donation page Experiment ID: #17705

Buckner International

Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.

Experiment Summary

Timeframe: 6/24/2019 - 8/15/2019

Based upon the low traffic volume to the Shoes for Orphan Souls donation page, we knew that we would need to try something radical in order to validate an experiment. With this in mind, we decided to launch a radical redesign of the page in conjunction with a similar experiment on Buckner’s main donation page. Both experiments tested similar elements:

  • We modified the form headers to be numbers steps instead of just labels.
  • We removed the gift array which we’ve seen help with organizations that have a large volume of mid-level and major donors
  • We increased the perceived security near the credit card form
  • We reiterated the gift size and impact near the donation button

Research Question

Will a radical redesign of the donation page increase overall donor conversion?

Design

C: Control
T1: With Updates

Results

Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 10.4% $0.00
T1: With Updates 20.3% 94.0% 89.9% $0.00

This experiment has a required sample size of 105 in order to be valid. Since the experiment had a total sample size of 141, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

While it did not statistically validate and it had a small number of conversions, we did see a directional lift with the radical redesigned donation page. What is surprising is that the experiment run on the main donation page that mirrored these changes saw a completely opposite result (41% decrease.)

When we analyzed both experiments, what we found was that the decrease on the main donation page was the result of a decrease in donors less than $100. The SOS donation page did not have this same issue (we actually saw an increase on this page.) The key difference is in the final subheadline on both donation pages. The SOS donation page makes an ask for $25 and the main donation page did not suggest any gift amount.

It is our hypothesis that removing the gift array from the main page increased the decision friction for the small-dollar donors. The SOS donation page did not experience the same impact since it had the $25 in the final call to action.


Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.