How promoting an offer in the Dear Reader ad copy instead of membership increased donor conversion rate Experiment ID: #18185
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Timeframe: 9/9/2019 - 9/14/2019
In previous experimentation with the “Dear Reader” ads for The Daily Signal, we had noticed that there was a consistent finding of 5x to 10x more click activity on the ad when we would change the call to action from “Activate your membership” to “Learn more.” This told us that there was a lot of additional click activity available to us that we were not capturing.
In an attempt to see if this would translate into increased giving and revenue, we experimented with promoting a new offer instead of membership in this ad for The Heritage Foundation.
Does promoting an offer instead of membership increase giving or revenue?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|C:||THF Membership Ask (Control)||0.05%|
|T1:||Offer Promotion (Treatment)||0.41%||803.8%||100.0%|
This experiment has a required sample size of 1,356 in order to be valid. Since the experiment had a total sample size of 18,442, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 803.8% increase in conversion rate
× 0% increase in average gift
With 100% level of confidence, we experienced a +803% increase in donor conversion rate with this treatment.
This conversion rate increase translated into an increase of revenue by +783% (also with a 100% level of confidence). We also noted a +1002% increase in ad clickthrough rate, as well.
It would appear that promoting an offer instead of membership is more effective at driving conversions.
Our thought is that this is because promoting membership to existing members was met with the outcome of no clicks because people thought: “I’m already a member, so I don’t need to click to activate my membership.” By promoting an offer instead, we capitalized upon this dormant audience and the instant donation page behind the offer download page provided an opportunity for additional giving to people who have not be clicking Dear Reader ads in the past.