Billy Graham Evangelistic Association

How hiding gift designations on a primary donation page impacts donor conversion

Experiment ID: #11923

Billy Graham Evangelistic Association

Experiment Summary

Timeframe: 05/03/2019 - 05/26/2019

On BGEA’s primary donation page, they present 14 different designations for people to choose from when making a gift. In looking at this we hypothesized that the significant array of options for people to choose from may be causing friction in the donation pathway. A person would need to look through all of the 14 giving options and select where they’d like to designate their gift towards before moving on to the next step in the donation process. The test our hypothesis, we hid all 14 designations on the donation page and added a link that people could click on if they wanted to see all of the options. 

Research Question

Will hiding the 14 gift designations increase donor conversion?

Design

C: Control
T1: Hidden Designations

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 45.1%$0.00
T1: Hidden Designations 40.9%-9.4% 99.9%$0.00

This experiment has a required sample size of 1,041 in order to be valid. Since the experiment had a total sample size of 5,498, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 9.4% decrease in conversion rate
× 0% increase in average gift

Key Learnings

To our surprise, hiding the gift designations decreased donor conversion. This result was true for every device type and user segment (new vs. returning). While we saw a negative impact on overall donor conversion, we hypothesize that we may have seen an increase in “where most needed” but a decrease from the other designations. We are dissecting the data now to look into this hypothesis. 
We also hypothesize that by eliminating all gift designations, we may have removed some of the most used designations resulting in the decrease in conversion. We will now test into this hypothesis in two ways: hiding most of the designations except for the top 3-5, and then also testing a “most popular” designation by visually setting this off and adding value proposition to it. 


Experiment Documented by Courtney Gaines
Courtney Gaines is Vice President at NextAfter.

Question about experiment #11923

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.