How hiding gift designations on a primary donation page impacts donor conversion Experiment ID: #16113
Billy Graham Evangelistic Association
Timeframe: 5/3/2019 - 5/26/2019
On BGEA’s primary donation page, they present 14 different designations for people to choose from when making a gift. In looking at this we hypothesized that the significant array of options for people to choose from may be causing friction in the donation pathway. A person would need to look through all of the 14 giving options and select where they’d like to designate their gift towards before moving on to the next step in the donation process. The test our hypothesis, we hid all 14 designations on the donation page and added a link that people could click on if they wanted to see all of the options.
Will hiding the 14 gift designations increase donor conversion?
|Treatment Name||Conv. Rate||Relative Difference||Confidence||Average Gift|
This experiment has a required sample size of 1,041 in order to be valid. Since the experiment had a total sample size of 5,498, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 9.4% decrease in conversion rate
× 0% increase in average gift
To our surprise, hiding the gift designations decreased donor conversion. This result was true for every device type and user segment (new vs. returning). While we saw a negative impact on overall donor conversion, we hypothesize that we may have seen an increase in “where most needed” but a decrease from the other designations. We are dissecting the data now to look into this hypothesis.
We also hypothesize that by eliminating all gift designations, we may have removed some of the most used designations resulting in the decrease in conversion. We will now test into this hypothesis in two ways: hiding most of the designations except for the top 3-5, and then also testing a “most popular” designation by visually setting this off and adding value proposition to it.