KCBI

How adding a monthly donate button to the main navigation menu impacted donor conversion

Experiment ID: #94883

KCBI

Experiment Summary

Timeframe: 05/23/2022 - 06/10/2022

KCBI is a large Christian radio station in the DFW area with a worldwide listening audience. They wanted to see if they could improve donor conversion on their website by adding a monthly donation button to their main navigation next to their one-time donate button. Monthly donors are an important part of their program and they hoped that this additional callout would increase monthly donor conversion.

Research Question

We believe that adding a monthly donate button to the main navigation menu will increase donor conversion

Design

C: Control
T1: Dual Donation Buttons

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.98% $0.00
T1: Dual Donation Buttons 2.3% 137.0% 98.7% $0.00

This experiment has a required sample size of 688 in order to be valid. Since the experiment had a total sample size of 2,231, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 137.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

Not only did the addition of a monthly donate button increase donor conversion, but it increased revenue as well by 138.3% with a 94% level of confidence. Our hypothesis is that donors appreciated the clarity of knowing the two types of donations they could make at the start of the process while also just simply giving them another visual reminder of how easy it is to donate. While we originally hoped to also validate an increase in monthly donor conversion, the small sample size of monthly donations was not sufficient for a reliable level of confidence.

Caveats. The language in the onetime donation option changed from the control to the treatment. This muddied the results of the test. Also by adding an additional button, we had to take the onetime gift button out of the nav menu with the other menu items and present it on its own.

Future Testing:

We need to ensure that the changes to the nav menu are rolled out to the entire KCBI site to increase sample size and gain more accurate data. With that in place, we can test the following elements from this test to learn what had the greatest impact:

  • Test position of the onetime gift button
  • Test the copy on the onetime gift button (give vs donate)
  • test adding a monthly donate button again once we have the data from the prior two tests.


Experiment Documented by NextAfter

Question about experiment #94883

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.