How adding a tabbed form on an acquisition offer impacted overall giving
The Gospel Coalition
Experiment Summary
Timeframe: 03/25/2022 - 04/29/2022
Because we have found recurring donors to consistently have a higher lifetime value as well as a higher retention rate than one time donors, we wanted to identify additional ways to convert donation page visitors into recurring donors. The standard donation form control has a checkbox underneath the gift array that says “Make this a monthly gift” but our hypothesis was that asking for the gift amount first was causing friction to then commit the chosen amount as a monthly gift after the fact. We wanted to test introducing a tabbed form on an acquisition offer so we could determine if a tabbed form would have any substantial negative impact on overall giving before we tried to put this option on the main TGC donation page. We split traffic 50/50 between the two pages, making only the change to the donation form presentation of one time vs. monthly giving.
Research Question
We believe that offering a tabbed form for instant donation page visitors will achieve an increase in monthly donations.
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
---|---|---|---|---|---|
C: | Control | 0.44% | $0.00 | ||
T1: | Treatment | 1.8% | 305.8% | 99.5% | $0.00 |
This experiment has a required sample size of 473 in order to be valid. Since the experiment had a total sample size of 1,861, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 305.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
Unfortunately, after running the test for one month, zero recurring donations were made on either the control or the treatment. However, we were still able to analyze the one-time donations on the two versions. What happened was a 305.8% increase in donor conversion rate on the treatment with 99.4% level of confidence. As a result, revenue also experienced a 253.3% increase with a 97% level of confidence on the treatment page. Lifts were consistent across both desktop and mobile visitors alike so we can move forward with testing the tabbed form experience in an attempt to increase recurring giving on the main donation form without concern that there will be a negative impact or decrease in overall conversion. On future tests, we will look for ways to highlight the value proposition for monthly giving in the body copy to continue to aim for increased recurring donors. It is important to note that we did not reach the desired minimum of 50 transactions to fully validate this test, so we will need to re-run this on another offer where we can get additional traffic and volume of conversions to know for sure if the treatment is producing a true lift.
Question about experiment #89457
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.