How does a tabbed donation form affect recurring conversion rates on a primary donation page?
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 01/10/2023 - 02/10/2023
We wanted to know how a tabbed donation form (featuring single and monthly gifts) would affect recurring donations on the primary donation page for a public policy organization.
In this experiment, the control presented the option to make a recurring gift by featuring a checkbox below the gift array that asked the visitor if they would like to make their gift on a recurring basis. Beneath the checkbox, an impact statement communicated the impact of making a recurring gift.
The treatment instead presented the option to make a recurring gift in a tabbed form format, with “single gift” and “monthly gift” shown side-by-side above the gift array and the impact statement shown above the tabs.
For this experiment, ~50% of traffic was shown the control and ~50% of traffic was shown the treatment.
Research Question
We believe that a tabbed donation form for donation page visitors will achieve higher recurring donation rates by making this giving option and its impact more visible.
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | $2.13 | $117.40 | ||
| T1: | Tabbed Donation Form | $15.26 | 616.3% | 94.4% | $109.31 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.
Key Learnings
After running this experiment for one month, we observed that the tabbed donation form produced an 11.0% decrease in recurring gifts with a 25% level of confidence, a 25.4% decrease in overall donations with a 99.2% level of confidence, and a 32.1% decrease in traffic with a 94.4% level of confidence.
These decreases were most severe among returning visitors where total donations decreased by 29.5% with a 99.7% level of confidence and revenue decreased by 40.6% with a 97.6% level of confidence indicating.
For new visitors, we observed a large increase in revenue (177%) — but because this increase was a result of just one additional donor, it is apparent that this is being skewed by a large gift and not indicative of overall performance.
These results disprove our hypothesis and indicate that in the context of visiting a primary donation page (high donor motivation), a tabbed donation form with the impact statement above the gift array may have distracted the donor or created friction by presenting multiple options before they made the decision to give.
It also suggests that it is more persuasive to ask someone to “upgrade” their donation after they have already made a micro-commitment by selecting their donation amount from the gift array.
Based on these results, we will look to test a monthly giving checkbox against the tabbed donation forms currently running on this organization’s instant donation pages tied to acquisition offers.