The Heritage Foundation

How a radical redesign impacted monthly giving for membership renewals

Experiment ID: #157263

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 05/09/2023 - 07/02/2023

For visitors looking to renew their membership from the homepage, we wondered if the existing donation page language for renewals was too focused on the organization. We developed a radical redesign language with customized value proposition for existing members who are already familiar with the organization and had shown historical motivation to make a gift. We hypothesized that increasing the value prop elements of appeal and exclusivity would frame a significantly more attractive way of describing what the user desires. Even if the internal goal is to get a user to renew their membership, the user wants to help protect our nation’s founding principles and keep their membership active. We wanted to communicate this clearly on the page to see if it would increase conversion. We updated the language to recognize that we were talking to visitors who had previously supported the organization and had arrived on this page to make an additional gift to support the organization. We also tested the presentation of the monthly ask, changing it from a checkbox after the gift amount had already been selected to a tabbed form with “Join the Leaders Club” as the text for monthly gifts.

Research Question

We believe that a radical redesign with customized value proposition for returning members looking to renew will achieve an increase in donor conversion because the enhanced appeal and exclusivity focus on what the user wants to do rather than the organization’s internal goals. because the enhanced appeal and exclusivity focus on what the user wants to do rather than the organization’s internal goals.

Design

C: Control
T1: Radical Redesign

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 1.6%$161.00
T1: Radical Redesign 3.2%97.7% 92.7%$98.24

This experiment has a required sample size of 713 in order to be valid. Since the experiment had a total sample size of 1,204, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

After running the experiment for two months, the radical redesign treatment of the donation page with a customized value proposition for returning members looking to renew resulted in a 97% increase in Leaders Club Member conversion with 92% level of confidence. Although the statistical level of confidence is below 95%, the results are directionally approaching validity. This suggests that the enhanced appeal and exclusivity focused on what the user wants to do rather than the organization’s internal goals successfully attracted more donors. The treatment version also removed an additional step that asked the visitor to select which area their donation should go towards, so this reduced friction could have also played a role in the increased conversion.

We also saw flat results for one-time donations on this page, so there is opportunity to isolate the value proposition copy and do further testing to increase one-time gifts as well as monthly. We believe future iterations could include additional clarity around the benefits of being a Leaders Club member in addition to the tabbed form design and title to help more visitors choose this option.

Overall, the key learning is to focus on the user’s desires and motivations, enhancing the value proposition in alignment with what we already know about the intended audience, in order to increase donor conversion rates.


Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #157263

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.