How a “most popular” pre-selected value in the gift array did not hurt conversion or revenue
The Fund for American Studies
Experiment Summary
Timeframe: 10/04/2019 - 11/04/2019
In an attempt to increase the average gift for The Fund for American Studies’ acquisition campaign, we decided to experiment with a “most popular” treatment on the $50 amount in the gift array to see if it would increase the average gift amount from instant donors.
Research Question
Would a visual treatment using that increases the visibility of the pre-selected value increase the average gift from instant donors?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 4.5% | ||
| T1: | Most Popular Treatment | 5.0% | 10.9% | 39.2% |
This experiment has a required sample size of 14,406 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Although we did not see a validated increase in either donor conversion rate or revenue, we observed a directional lift in both metrics.
The key learning with this experiment was that after running for a month, this treatment DID NOT show, at any point, a decrease in either one of these metrics, which was enough for us to consider that the lifts we achieved were directional, and that there would not be risk in rolling this treatment out as the “new control” version of the donation page.