How a visual redesign of a main donate button on the website impacted clicks
The Gospel Coalition
Experiment Summary
Timeframe: 01/25/2022 - 03/03/2022
The TGC team was inspired by another experiment they saw in a NextAfter Institute webinar to visually modify the main donate button on their website navigation bar. Rather than rolling out the new version based solely on another organization’s results, we decided to test into the optimal design by splitting traffic 50/50 and measuring the results of how many people clicked on each version of the button. The existing button said “Give” and was visually the same as the other calls to action in the navigation header. The treatment was a green button to help it stand out visually from the rest of the page and the text was revised to “Donate to TGC” to add clarity.
Research Question
We believe that visually emphasizing the donate button for homepage visitors will achieve an increase in clicks to the donation page.
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
---|---|---|---|---|---|
C: | Old - Give Button on Website | 0.03% | $0.00 | ||
T1: | New - Donate to TGC Green Button | 0.04% | 27.8% | 99.7% | $0.00 |
This experiment has a required sample size of 335,414 in order to be valid. Since the experiment had a total sample size of 1,544,797, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 27.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running the experiment for one month, we saw a valid increase in clicks on the new green button that said “Donate to TGC”. What was interesting, however, is that donations and revenue recorded on the donation page were actually higher coming from the control version. Because an experiment was simultaneously running on the main donation page that could have been responsible for impacting these results, we will have to take this into account on a future experiment. Overall we learned that the new button design did, in fact, attract significantly more visitors to the main donate page.
Question about experiment #83279
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.