How adding a monthly donate button to the main navigation menu impacted donor conversion
KCBI
Experiment Summary
Timeframe: 05/23/2022 - 06/10/2022
KCBI is a large Christian radio station in the DFW area with a worldwide listening audience. They wanted to see if they could improve donor conversion on their website by adding a monthly donation button to their main navigation next to their one-time donate button. Monthly donors are an important part of their program and they hoped that this additional callout would increase monthly donor conversion.
Research Question
We believe that adding a monthly donate button to the main navigation menu will increase donor conversion
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.98% | $0.00 | ||
| T1: | Dual Donation Buttons | 2.3% | 137.0% | 98.7% | $0.00 |
This experiment has a required sample size of 688 in order to be valid. Since the experiment had a total sample size of 2,231, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 137.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
Not only did the addition of a monthly donate button increase donor conversion, but it increased revenue as well by 138.3% with a 94% level of confidence. Our hypothesis is that donors appreciated the clarity of knowing the two types of donations they could make at the start of the process while also just simply giving them another visual reminder of how easy it is to donate. While we originally hoped to also validate an increase in monthly donor conversion, the small sample size of monthly donations was not sufficient for a reliable level of confidence.
Caveats. The language in the onetime donation option changed from the control to the treatment. This muddied the results of the test. Also by adding an additional button, we had to take the onetime gift button out of the nav menu with the other menu items and present it on its own.
Future Testing:
We need to ensure that the changes to the nav menu are rolled out to the entire KCBI site to increase sample size and gain more accurate data. With that in place, we can test the following elements from this test to learn what had the greatest impact:
- Test position of the onetime gift button
- Test the copy on the onetime gift button (give vs donate)
- test adding a monthly donate button again once we have the data from the prior two tests.