How expressed membership levels impacts donor conversion for existing members Experiment ID: #8786

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 3/27/2018 - 4/3/2018

For the final days of the March membership campaign, The Heritage Foundation wanted to see if they could upgrade and incentivize giving by explicitly identifying the membership levels and associated benefits with each gift amount. We decided to do this at the gift array level for clarity and we wanted to highlight a higher giving level to incentivize an increased average gift. We then tested this against the existing control which is a standard giving array that is variable based upon a member’s last gift.

Research Question

Will explicitly calling out membership levels on the giving array impact donor conversion?

Design

C: Standard Gift Array
T1: Defined Levels

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Standard Gift Array 64.9%
T1: Defined Levels 56.8% -12.4% 99.8%

This experiment has a required sample size of 278 in order to be valid. Since the experiment had a total sample size of 1,522, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 12.4% decrease in conversion rate
× 0% increase in average gift

Key Learnings

When targeting existing members, we found that the landing page that emphasized the various membership levels actually decreased overall conversion by 12%. Additionally, it was also shown to decrease the average gift by $12. It is our hypothesis that this treatment did not work because it focused on the benefits of the various membership levels and past experiments have shown existing members are not as driven by benefits as nonmembers. Based upon that hypothesis, we do believe it is worth testing this with nonmembers for acquisition.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.