How using “lowest click” settings for donation remarketing decreased donor conversion rate Experiment ID: #17483

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 7/31/2019 - 8/28/2019

In an attempt to experiment with what drives remarketing gifts for the recent Board Challenge campaign for The Heritage Foundation, we decided to experiment with attempting to drive more click volume by targeting remarketing ads based upon “lowest cost per click” value against the “purchase” optimization ad serving method.

Our hypothesis is that if someone entered into the remarketing ads campaign, they clicked an email to go to the campaign landing page. So, instead of paying a premium to serve remarketing ads to those who are only most likely to “purchase” based upon Facebook’s algorithm, but rather try to get as many people back to the donation page that saw it before, in an attempt to secure more donations overall.

Research Question

Will the “lowest cost per click” setting increase donor conversion rate for Facebook Remarketing campaigns over just using the “Purchase” optimization method?

Design

C: 2019 Board Challenge | Remarketing Campaign | Purchase
T1: 2019 Board Challenge | Remarketing Campaign | Click

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: 2019 Board Challenge | Remarketing Campaign | Purchase 0.10%
T1: 2019 Board Challenge | Remarketing Campaign | Click 0.00% -100.0% 99.5%

This experiment has a required sample size of 2,930 in order to be valid. Since the experiment had a total sample size of 19,729, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 100.0% decrease in conversion rate
× 0% increase in average gift

Key Learnings

With 99.5% level of confidence, we determined that the “lowest cost per click” remarketing target approach decreased donor conversion rates by -100%.

Below outlines the other observations we made between the two campaign types, as well:

  • Impressions: The “lowest cost” (treatment) campaign increased the number of ad impressions by +54%.
  • Reach: The “purchase” optimized campaign (control) increased reach by +10%.
  • Clicks: The “lowest cost” (treatment) campaign increased the volume of clicks by +8%.
  • Cost: The “purchase” campaign was far more costly, spending +399% more of the budget when compared to the “lowest cost” (treatment) campaign.

Final Summary & Recommendation:

Although the “purchase” optimized campaign was 4 times more expensive when compared to the “lowest cost per click” setting for remarketing ad campaigns on Facebook, it was infinitely more effective at identifying people in the list that were most likely to convert, serve ads to them, then successfully bring them back to convert for the campaign.

For this reason, we would always advocate for the “purchase” optimization event to be used for remarketing campaign types, as the increased expense will generate increased results when compared to attempting to tell Facebook to spend your budget in the most frugal of ways.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.