How expressed membership levels impacts donor conversion for existing members
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 03/27/2018 - 04/03/2018
For the final days of the March membership campaign, The Heritage Foundation wanted to see if they could upgrade and incentivize giving by explicitly identifying the membership levels and associated benefits with each gift amount. We decided to do this at the gift array level for clarity and we wanted to highlight a higher giving level to incentivize an increased average gift. We then tested this against the existing control which is a standard giving array that is variable based upon a member’s last gift.
Research Question
Will explicitly calling out membership levels on the giving array impact donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Standard Gift Array | 64.9% | ||
| T1: | Defined Levels | 56.8% | -12.4% | 99.8% |
This experiment has a required sample size of 278 in order to be valid. Since the experiment had a total sample size of 1,522, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 12.4% decrease in conversion rate
× 0% increase in average gift
Key Learnings
When targeting existing members, we found that the landing page that emphasized the various membership levels actually decreased overall conversion by 12%. Additionally, it was also shown to decrease the average gift by $12. It is our hypothesis that this treatment did not work because it focused on the benefits of the various membership levels and past experiments have shown existing members are not as driven by benefits as nonmembers. Based upon that hypothesis, we do believe it is worth testing this with nonmembers for acquisition.