How a visual match reinforcement affects year-end giving
The Fund for American Studies
Experiment Summary
Timeframe: 12/10/2021 - 01/01/2022
For TFAS’ calendar-year-end campaign, we wanted to create a visual reinforcement for the matching gift. The goal is to have a visual representation of how a donor’s impact is doubled by the matching gift when they give during the year-end campaign. In this particular case we used a numerical visual representation that would instantly change as the prospective donor selected a different amount in the gift array. When someone selected $50, the counter would show a $100 impact. If $100 was selected, the counter would display a $200 impact, etc. We hypothesized that this visual representation would increase both donor conversion rate and revenue.
Research Question
We believe that providing a visual representation for a matching gift for prospective donors will achieve a higher donor conversion rate and revenue.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 4.4% | $0.00 | ||
| T1: | Match Counter | 4.9% | 11.6% | 22.9% | $0.00 |
This experiment has a required sample size of 13,228 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Though this experiment did not fully validate, there was a directional decrease and enough evidence to indicate that this visual representation did not result in a higher donor conversion rate, nor did it inspire a higher average gift. There was a 62% directional decrease in revenue through this experiment. The speculation is that the visual number doubling, may have been confusing for some prospective donors and perhaps felt like they were donating $100 instead of $50. Another way we can test this concept in the future is to test the doubling of the gift through a visual representation on a thermometer instead.
Notable changes to other metrics due to the experiment:
- Revenue experienced a 62.8% decrease with a 92% level of confidence.
Observations of the impact on other visitor segments in the experiment:
- Email visitors had a 53.5% decrease in donations with a 98% level of confidence.
- New visitors had a 54.9% decrease in donations with a 93% level of confidence.
- Mobile visitors had an 84.4% decrease in donations with a 100% level of confidence.
- Referral visitors had a 100.0% increase in donations with a 100% level of confidence.
- Male visitors had an 87.5% decrease in donations with a 100% level of confidence.