How a tabbed donation form affects recurring giving rates on an acquisition offer
American Cornerstone Institute
Experiment Summary
Timeframe: 10/27/2022 - 12/02/2022
In an effort to increase the recurring giving rate on an instant donation page of an acquisition offer, we wanted to know how increasing the visibility of the monthly giving option earlier in the donation process, with a tabbed donation form (which in the past has shown success in this regard), would affect recurring monthly giving.
We ran this experiment for 30 days as an A/B test splitting traffic 50/50 between:
A. the control that features a monthly giving checkbox beneath the gift array, and
B. the treatment that removes the checkbox in favor of a monthly giving tab above the gift array
In addition, the impact statement featured below the monthly giving checkbox in the control was moved above the monthly donation form in order to keep it in close proximity to the monthly giving option.
Research Question
We believe that a tabbed donation form will increase recurring giving rates on the donation page of an acquisition offer by increasing the visibility of the option to give monthly.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.46% | $0.00 | ||
| T1: | Tabbed Donation Form | 0.38% | -16.6% | 47.8% | $0.00 |
This experiment has a required sample size of 55,886 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
After running this experiment for 30 days, the treatment featuring the tabbed donation form produced a 16.6% DECREASE in recurring giving rates with a 47.7% LoC. While the low statistical validity of this experiment prevents us from drawing definitive conclusions about these results one way or another, we did observe that perhaps the checkbox in the control may have offered a more logical “flow” during the donation process, prompting users to first select their donation amount, then select how often they want to give, then view the impact that monthly donation will make.
But because the results of this experiment were well below the 95% LoC that would make them statistically valid, and because we have seen tabbed donation forms produce lifts in recurring giving for other clients in the past, we will want to look for opportunities to test this treatment on a donation page that has an opportunity to produce a larger sample size and validate the results one way to the other.