How a Monthly Pop-Up on a Main Donation Page Impacts Donor Conversion
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Timeframe: 01/03/2024 - 01/12/2024
To increase monthly gifts on CaringBridge’s main donation page, we hypothesized a pop-up that presents the user with an opportunity to turn their original one-time amount into a lowered monthly gift would increase recurring donations. This pop-up would appear when the user submitted their one-time donation. We hypothesized that by making this conversion easy, suggesting a lower amount than what the user was originally going to donate, and reinforcing the impact of that smaller amount on a consistent basis it would improve conversion rate.
Research Question
We believe that presenting a monthly popup with a lowered gift amount for donors submitting their one-time donation will achieve an increase in monthly gifts because the popup reinforces the impact of a smaller amount than their original gift made on a consistent basis. .
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 8.5% | |||
| T1: | Serial Pop up Ask | 6.3% | -25.0% | 77.2% |
This experiment has a required sample size of 1,180 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The key learnings from this experiment are as follows:
The treatment, which involved presenting a monthly pop-up with a lowered gift amount for donors submitting their one-time donation, resulted in a 25.0% decrease in donations for all traffic. However, the statistical confidence level of 77.2% is below the generally accepted threshold of 85%, so these results cannot be considered valid.
Our hypothesis for why this popup not only didn’t increase monthly donations but in fact decreased donations overall is due to the way CaringBridge’s form is preset. Monthly is already pre-selected as the default giving option, so to submit a one-time gift a user would have to intentionally click it, fill out the form, and press submit. Targeting that audience, who already has a specific motivation for making a one-time donation, with another popup likely made the experience more disruptive rather than presenting them with something they hadn’t already considered.
In future testing, we will focus our efforts on that initial action — selecting one-time rather than monthly — at the top of the form so that we are meeting the donor at the point of consideration rather than at the end where they’ve already made a decision.