The Heritage Foundation

How point-of-donation upgrading affects conversion

Experiment ID: #780

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 01/11/2015 - 01/23/2015

For existing members to the Heritage Foundation, the donation form is prepopulated with the donor’s prior giving amount as a suggested gift.  However, in the case of the control, it is not clearly presented to the member what level their donation qualifies them for and what benefits are provided.  As a way to upgrade these donors, this experiment contemplated the value of communicating the membership level and associated benefits for a donation at the next highest level of membership.

Research Question

Do members to the Heritage foundation give more in order to receive greater benefits?  How does communicating the available benefits of the next level affect conversion and average gift amount?


C: No Membership Language
T1: Membership Language on the Form


 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: No Membership Language 39.0%$69.00
T1: Membership Language on the Form 34.1%-12.6% 100.0%$77.41

This experiment has a required sample size of 742 in order to be valid. Since the experiment had a total sample size of 6,745, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 12.6% decrease in conversion rate
× 12.2% increase in average gift
1.9% decrease in revenue

Key Learnings

The results of this experiment are very interesting.  On one hand, including the upgrade language increases average gift amount by 12.2%.  But on the other hand, the upgrade language decreased average gift by 12.6%.  This suggests that though the tactic may prove to be effective to get a donor to upgrade their gift amount, it may also turn others off.  More testing needs to be conducted to understand how changes in the language might be able to mitigate the drop off in conversion while still positively affecting average gift amount.

Experiment Documented by Tim Kachuriak
Tim Kachuriak is Chief Innovation and Optimization Officer of NextAfter.

Question about experiment #780

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.