How increasing the gift array on a donor acquisition page impacted overall revenue Experiment ID: #11941

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Ended On: 2/19/2019

Continuous testing to improve revenue from clickthrough traffic from The Daily Signal’s “Dear Reader” visitors has yielded consistent wins by the control. At this point, we wanted to experiment with the gift array to see if we could increase the revenue by just simply asking for a higher gift amount.

Research Question

Will increasing the gift array improve donor conversion rates and revenue?

Design

C: $25, $50, $100
T1: $50, $75, $100

Results

Treatment Name Revenue per Visitor Relative Difference Confidence Average Gift
C: $25, $50, $100 $0.03 $41.03
T1: $50, $75, $100 $0.01 -63.1% 99.8% $42.28

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 64.2% decrease in conversion rate
× 3.0% increase in average gift
63.1% decrease in revenue

Key Learnings

With a 99.7% level of confidence, we validated a decrease of revenue by 63%. We also validated (with a 99% LoC) that a higher gift array (starting at $50) decreased the donor conversion rates on this page by 64%, as well.

Meanwhile, the increase in average gift for the treatment did increase, but by only a negligible amount.

Typically, we would want to collect a sample of at least 100 donations, but we pulled the plug because the revenue disparity was so large between the two treatments.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.