How a “most popular” pre-selected value in the gift array did not hurt conversion or revenue Experiment ID: #18716
The Fund for American Studies
Timeframe: 10/4/2019 - 11/4/2019
In an attempt to increase the average gift for The Fund for American Studies’ acquisition campaign, we decided to experiment with a “most popular” treatment on the $50 amount in the gift array to see if it would increase the average gift amount from instant donors.
Would a visual treatment using that increases the visibility of the pre-selected value increase the average gift from instant donors?
|Treatment Name||Conv. Rate||Relative Difference||Confidence||Average Gift|
|T1:||Most Popular Treatment||5.0%||10.9%||39.2%||$0.00|
This experiment has a required sample size of 14,406 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Although we did not see a validated increase in either donor conversion rate or revenue, we observed a directional lift in both metrics.
The key learning with this experiment was that after running for a month, this treatment DID NOT show, at any point, a decrease in either one of these metrics, which was enough for us to consider that the lifts we achieved were directional, and that there would not be risk in rolling this treatment out as the “new control” version of the donation page.