The Heritage Foundation

How positioning the Membership Ask as the best way to ally with the President impacted donor conversion rates for The Daily Signal readers.

Experiment ID: #10364

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 01/03/2019 - 01/09/2019

After the “Trump Ask” did so well during calendar year end fundraising at converting The Daily Signal traffic into members for The Heritage Foundation, we decided to take the same message and roll it out against the standard “Dear Reader” membership ask.

Research Question

Was the “Trump Ask” message so effective during calendar year end fundraising just because motivation is always higher during that time of the year, or does positioning our membership ask as the best way to ally with President Trump truly more effective at converting donors?

Design

C: Membership
T1: CYE Treatment

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Membership 10.8%
T1: CYE Treatment 22.4%107.0% 100.0%

This experiment has a required sample size of 74 in order to be valid. Since the experiment had a total sample size of 675, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 107.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 99.9% confidence score, we can definitively say that the treatment increased donor conversion rates by 107% over the standard control membership ask in the “Dear Reader” ads on The Daily Signal.

Although the treatment generated 39% less visitors to the donation page, it still generated 20.6% more revenue, 88% lift in revenue/visitor, with a negligible difference between the two treatments in average gift (-6.4%).

It’s probably important to also note that the relevance of the message likely played an important role in the performance of this treatment. For future reference, the recent mid-term elections saw the control of the House of Representatives flip from Republicans to Democrats, and this president has been seen as divisive. The treatment message positions a membership at Heritage as “the best way to ally with the president” — as he has been “under attack” in the media and from political adversaries probably more than any other president in our country’s history.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #10364

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.