How simplifying a gift array impacted lapsed donor reactivation rate
Canadian Taxpayers Federation
A not-for-profit citizen's advocacy group dedicated to lower taxes, less waste and accountable government.
Experiment Summary
Timeframe: 11/22/2022 - 12/12/2022
We were running a survey to an audience of lapsed donors and followed the survey with an instant donation page. In previous testing, we had found that asking for a $10 gift in the body of the email copy increased donor conversion rate. We were interested in testing this same concept at the donation page level and hypothesized that reducing the number of choices on the gift array to only ask for a $10 gift would increase conversion rate from these lapsed donors. We removed all choices other than “$10” and “Other” from the gift array and split traffic equally between the control and the treatment.
Research Question
We believe that isolating a $10 ask on the gift array for lapsed donor survey takers will achieve an increase in donor conversion.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 13.9% | $0.00 | ||
| T1: | only $10 | 25.0% | 79.8% | 99.6% | $0.00 |
This experiment has a required sample size of 99 in order to be valid. Since the experiment had a total sample size of 407, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 79.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
What we observed after running the experiment for 2 1/2 weeks was that there was a 79.8% increase in conversion rate with 99.5% level of confidence. We believe this is due to the reduced friction in the treatment experience where the donor did not have to pause and decide on an amount to give they could just decide “Do I want to give $10? Yes or no?” and continue their cognitive momentum from answering questions on the survey. We made the choice simple, reducing the perceived barrier in the decision-making process. While this treatment produced a better response in reactivating lapsed donors, in future testing we would like to test how this might impact new donor conversion as well. We also want to test increased gift amounts (such as $15 or $20) on the single gift array option to identify if it is the single choice or the lower amount that impacts results.