How adding a donate button to a news platform site’s header affects donor conversion rate
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 04/02/2024 - 04/10/2024
The Daily Signal is The Heritage Foundation’s multimedia news platform. On their home page, and most other pages, the only call-to-action to donate to support the organization is located at the bottom in the footer section among many other site links.
In an effort to increase the amount of traffic to the donation page from the home page, we previously tested adding a more prominently displayed donate button to the category menu at the top of the page on the desktop version, and in the navigation menu of the mobile version. This experiment resulted in a 42.8% increase in donations compared to the control. However, with a 53.2% level of confidence, these results can’t be relied upon to predict a valid outcome.
In this attempt to increase traffic to the donation page, we wanted to test whether adding the donate button to a more notable location at the very top of the page would lead to an increase in donor conversion rate. We also wanted to test whether adding a gift box icon to the donate button in order to match the “branded” style of the “Search” and “Menu” buttons next to it would lead to an increase in conversion rate.
The regular donate button and “branded” donate button appeared in the top header sitewide for both the desktop and mobile experiences. 50% of traffic saw the control with no button, and the two treatments received 25% of traffic each.
Research Question
We believe that adding a donate button to the top header for site visitors will achieve an increase in donor conversion rate because more people will click to view the donation page.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.00% | $36.36 | ||
| T1: | Donate Button | 0.04% | 2,694.0% | 100.0% | $50.00 |
| T2: | Branded Donate Button | 0.04% | 2,866.2% | 100.0% | $25.00 |
This experiment has a required sample size of 15,324 in order to be valid. Since the experiment had a total sample size of 146,671, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
2,694.0% increase in traffic
× 0% increase in conversion rate
× 37.5% increase in average gift
3,741.7% increase in revenue
Key Learnings
The key learning from this experiment is that adding a donate button to the top header for site visitors led to a significant increase in traffic to the donation page. The first treatment saw a significant 2,694% increase in clicks to the donation page compared to the control group with no button, while the second treatment saw an even higher 2,866.2% increase in clicks at a 99.9% level of confidence. This result suggests that making the donation button more prominent and easily accessible can greatly impact the amount of traffic led to the donation page.
However, in terms of donations, the first treatment lead to a 9.3% decrease in donor conversion rate (at an invalid 14.5% level of confidence), and the second treatment led to a near-valid 64% decrease in donor conversion rate compared to the control.
While invalid, these results may suggest that by more prominently displaying the donate button and increasing traffic to the donation page, people who do click on the button may be less motivated to give than someone in the control experience who seeks out the donation page in order to support the organization.
Future experimentation is needed to understand what motivates visitors of The Daily Signal to make a gift to support their work. Experiment ideas include testing alternative copy on the donation page to motivate more viewers of the site to give, and testing an alternate icon that aligns more strongly with the organization’s mission.