The Heritage Foundation

How an expanded value proposition impacted member renewal

Experiment ID: #5340

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 03/24/2016 - 04/26/2016

For the member renewal campaign, we wanted to see if we could add some relevance to the appeal by talking about the upcoming challenges. We tested this expanded value proposition against the original control that had worked in the previous campaign.

Research Question

Which value proposition will result in the best conversion rate?


C: Control
T1: Expanded Value Prop


 Treatment NameRevenue per VisitorRelative DifferenceConfidenceAverage Gift
C: Control $35.08$78.91
T1: Expanded Value Prop $33.15-5.5% 74.5%$73.28

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.

Key Learnings

There was no statistically significant difference in donor conversion or average gift. This would seem to imply that the two different value propositions were not different enough to have much of an impact on a donor’s likelihood to give.

Experiment Documented by NextAfter

Question about experiment #5340

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.