How adding an average gift amount and minimum amount to the gift array impacts donor conversion rate
The Fund for American Studies
Experiment Summary
Timeframe: 09/02/2021 - 11/14/2021
The Fund for American Studies wanted to execute a test previously inspired by a wikipedia tests that states the average donation amount (which is often a number that is not a standard, round number), and a minimum donation. The minimum donation is determined by the number of visitors to the site and the goal revenue amount for the organization. It is expected that average gift would be lower in this experiment, but it was hypothesized that the increase in donor conversion would make up for the potential loss in revenue.
This specific experiment was targeted to traffic from a specific offer that generated the most traffic (Marx Survey). Most traffic to this page is from paid social.
Research Question
We believe that adding a minimum amount and average amount to the gift array for survey takers will achieve an increase in donor conversion .
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control - Default Gift Array | 1.9% | |||
| T1: | Treatment - Minimum amount & Average amount in Gift Array | 2.5% | 29.7% | 95.5% |
This experiment has a required sample size of 5,095 in order to be valid. Since the experiment had a total sample size of 10,671, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 29.7% increase in conversion rate
× 0% increase in average gift
Key Learnings
The final results of the experiment did provide a valid 29.7% lift in donor conversion. As expected, the average gift for the treatment was lower than that of the control. Our conclusion is that given amounts that make it relative to others’ giving habits, make prospective donors more likely to take action. However, it took several months for this experiment to validate. Should we roll this out as the control, we would need more data to ensure this is a viable strategy. It is recommended to test this on multiple instant donation pages as well as the main donation page.
Observations of the impact on other visitor segments in the experiment:
- Mobile visitors had a 31.1% increase in donations with a 94% level of confidence.
- Returning visitors had a 33.9% increase in donations with an 90% level of confidence.