How calling out a monthly gift on a main donation page affects recurring donor conversion
MAF
Experiment Summary
Timeframe: 08/22/2022 - 10/21/2022
Mission Aviation Fellowship wanted to test adding a “callout” to the recurring giving option on their main donation form to see if it would increase the number of recurring donations instead of one-time donations.
We added the callout to their main donation page for 50% of the traffic. The callout was a short, one-sentence description on how monthly giving has the greatest impact year-round.
Research Question
We believe that adding a monthly callout to the recurring option on the donation form for potential donors visiting the main donation page will achieve a higher recurring donor conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.66% | ||
| T1: | Monthly Callout | 3.7% | 455.5% | 98.9% |
This experiment has a required sample size of 178 in order to be valid. Since the experiment had a total sample size of 599, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 455.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
This experiment saw a 455% lift in monthly donor conversion with a 98.9% level of confidence on mobile devices specifically. We believe this happened because the callout absorbs a lot of white space on the mobile view making the recurring donation option stand out more to the user on this page compared to the control donation page or to the treatment page on a desktop.
Other notable takeaways that we saw in this experiment were a 33.9% lift in conversion rate for the all-traffic segment and a 28% lift in revenue for the all-traffic segment. Although neither one of those segments was validated, we believe the lift in conversion for both all-traffic segments can be attributed to the extra boost of value proposition that the donor was given when they were in the process of making their gift. Even though the callout was for monthly giving, it served as an extra piece of confirmation that a gift to Mission Aviation Fellowship was helping isolated people around the world. Therefore we believe the extra reminder within the process convinced more people to make a recurring gift and/or increase the size of their gift.
The last notable takeaway from this experiment is that this callout did not negatively affect donations altogether. We did not want to make users believe that a one-time gift was less impactful than a recurring gift, and it seems from the data, the callout did not have this impact. Donations for all giving only had 3.5% lift in conversion rate with a 34% level of confidence.
To continue further testing, we need to run an altered experiment for desktop users only.