How redesigning the main donate button on the homepage affected traffic at year-end
The Gospel Coalition
Experiment Summary
Ended On: 12/30/2021
Leading up to calendar year-end, we know that visitors to the TGC homepage had a high motivation to give. We wanted to make sure that potentially highly motivated donors could easily find where to make their year-end gift.
We rolled out a new button design on the homepage to see if we could make it easier for potential donors to find the donation page, but we wanted to run this as a test against the old version to capture the impact.
The new button (running as the control for this experiment) read “Donate to TGC” and we made it green to stand out for visual emphasis while the old button (the treatment version) simply said “Give” and visually matched all other menu options in the header.
We split traffic to each version.
Research Question
We believe that emphasizing the main donation button for homepage visitors will achieve an increase in traffic to the donation page.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Old - Give Button | 0.37% | ||
| T1: | New - Green Donate to TGC Button | 0.46% | 25.3% | 100.0% |
This experiment has a required sample size of 36,495 in order to be valid. Since the experiment had a total sample size of 410,520, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
25.3% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
What we found was the new button (green button) drove more traffic to the donation page. Of these two buttons, the “Donate to TGC” button provided the additional clarity necessary to increase traffic to the page while the white “Give” button saw a decrease in traffic of 20%.
Because we were running a separate test on the donation page, we did not factor in conversion rate or revenue on the donation page from this experiment. We only wanted to measure the impact on clicks through to the donation page. What we learned is that the more visual button made it easier for potential donors to find the next step we want them to take by adding clarity in the messaging and visually leading them where to go.