How increasing the gift array on a donor acquisition page impacted overall revenue
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Ended On: 02/19/2019
Continuous testing to improve revenue from clickthrough traffic from The Daily Signal’s “Dear Reader” visitors has yielded consistent wins by the control. At this point, we wanted to experiment with the gift array to see if we could increase the revenue by just simply asking for a higher gift amount.
Research Question
Will increasing the gift array improve donor conversion rates and revenue?
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | $25, $50, $100 | $0.03 | $41.03 | ||
| T1: | $50, $75, $100 | $0.01 | -63.1% | 99.8% | $42.28 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 64.2% decrease in conversion rate
× 3.0% increase in average gift
63.1% decrease in revenue
Key Learnings
With a 99.7% level of confidence, we validated a decrease of revenue by 63%. We also validated (with a 99% LoC) that a higher gift array (starting at $50) decreased the donor conversion rates on this page by 64%, as well.
Meanwhile, the increase in average gift for the treatment did increase, but by only a negligible amount.
Typically, we would want to collect a sample of at least 100 donations, but we pulled the plug because the revenue disparity was so large between the two treatments.