The Gospel Coalition

How a visual redesign of a main donate button on the website impacted clicks

Experiment ID: #83279

The Gospel Coalition

Experiment Summary

Timeframe: 01/25/2022 - 03/03/2022

The TGC team was inspired by another experiment they saw in a NextAfter Institute webinar to visually modify the main donate button on their website navigation bar. Rather than rolling out the new version based solely on another organization’s results, we decided to test into the optimal design by splitting traffic 50/50 and measuring the results of how many people clicked on each version of the button. The existing button said “Give” and was visually the same as the other calls to action in the navigation header. The treatment was a green button to help it stand out visually from the rest of the page and the text was revised to “Donate to TGC” to add clarity.

Research Question

We believe that visually emphasizing the donate button for homepage visitors will achieve an increase in clicks to the donation page.

Design

C: Old - Give Button on Website
T1: New - Donate to TGC Green Button

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Old - Give Button on Website 0.03%$0.00
T1: New - Donate to TGC Green Button 0.04%27.8% 99.7%$0.00

This experiment has a required sample size of 335,414 in order to be valid. Since the experiment had a total sample size of 1,544,797, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 27.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

After running the experiment for one month, we saw a valid increase in clicks on the new green button that said “Donate to TGC”. What was interesting, however, is that donations and revenue recorded on the donation page were actually higher coming from the control version. Because an experiment was simultaneously running on the main donation page that could have been responsible for impacting these results, we will have to take this into account on a future experiment. Overall we learned that the new button design did, in fact, attract significantly more visitors to the main donate page.


Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #83279

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.