How a tabbed donation form affects donor conversion rates on a primary donation page
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 11/27/2023 - 12/12/2023
We previously tested how a tabbed donation form affected recurring donations on a public policy organization’s main donation page. In this experiment, the tabbed donation form produced an 11% decrease in recurring gifts (with a 25% level of confidence) and a 25.4% decrease in overall donations (94.4% level of confidence), which disproved our hypothesis that a tabbed donation form would achieve higher recurring donation rates than a monthly giving checkbox by making the monthly giving option and its impact more visible.
In an effort to learn whether a tabbed donation form on this organization’s main donation page would lead to an increase in donor conversion rate, we retested this idea. The control in this experiment presents a monthly giving option as a checkbox below the gift array, while the treatment has two tabs for either a Single Gift or Monthly Gift with a value proposition callout box above the monthly gift tab. 50% of traffic saw the control and 50% saw the treatment.
Research Question
We believe that a tabbed donation form for main donation page visitors will achieve an increase in overall donor conversion rates.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.13% | $310.27 | ||
| T1: | Tabbed Donation Form | 0.26% | 99.5% | 58.5% | $347.14 |
This experiment has a required sample size of 8,909 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
After running this experiment for 16 days, we observed that the tabbed donation form on the main donation page led to an insignificant 2.2% increase in overall donor conversion rates, but its 11.3% level of confidence is not high enough to consider these results valid. Therefore, we cannot confidently conclude that the tabbed donation form is more effective in increasing conversions, but we also cannot conclude that it is less effective.
Additionally, the treatment saw a 99.5% increase in recurring donations, but at an invalid 58.4% level of confidence.
These results, along with data of tabbed donation forms leading to an increase in donations in the organization’s acquisition offers funnels, may indicate that people visiting their main donation page may have high donor motivation and be less influenced or even more distracted by the tabbed design of the form. The results also may have been impacted by an additional variable—the control not having the value proposition callout box found in the treatment’s experience.