How decision friction impacts donor conversion
Billy Graham Evangelistic Association
Experiment Summary
Timeframe: 06/28/2019 - 07/28/2019
On BGEA’s primary donation page, they offer people 14 different gift designations to choose from during the first step in their donation process. In an effort to optimize the giving experience and increase donor conversion, we previously tested removing all of the gift designation hypothesized narrowing down the number of gift designation options to the top five. Using historical giving data, we pulled the top five designations and developed a treatment that featured just those five. People would still be able to see the other 9 options if they wanted by clicking the “See more>>” link. We also developed another variation of the treatment that emphasized the top giving option as the “Most Urgent” need and removed the imagery and copy from the rest of the gift designations.
Research Question
Would we be able to increase donor conversion by removing some of the gift designations?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 42.5% | |||
| T1: | Top 5 Designation List | 47.3% | 11.4% | 99.9% | |
| T2: | Top 5 List with Urgent Need | 42.0% | -1.1% | 24.7% |
This experiment has a required sample size of 1,008 in order to be valid. Since the experiment had a total sample size of 7,028, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 11.4% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running this experiment for a month, we were able to increase overall donor conversion by 11.4% by narrowing down the gift designation options down from 14 to 5. Since this experiment was run on the primary donation page where motivations and audience can vary greatly, we analyzed all devices (mobile and desktop) and user types (new and returning). Regardless of device or user, donor conversion increased with the first treatment. The second treatment had differing results based on device and audience, but none of those results had high validity.
This experiment tells us that there was decision friction on the page in the number of designations and options we were presenting people. By reducing the numbers of options, we were able to increase donor conversion and overall revenue for the organization.
This page gets a significant volume of traffic to it. The potential impact that this experiment can have on revenue moving forward is substantial. In fact, based on historical giving, when the winning treatment is rolled out to 100% of the traffic over the next three months, it has the potential to bring in an additional projected $50k in revenue!