The Heritage Foundation

How using “lowest click” settings for donation remarketing decreased donor conversion rate

Experiment ID: #19041

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 07/31/2019 - 08/28/2019

In an attempt to experiment with what drives remarketing gifts for the recent Board Challenge campaign for The Heritage Foundation, we decided to experiment with attempting to drive more click volume by targeting remarketing ads based upon “lowest cost per click” value against the “purchase” optimization ad serving method.

Our hypothesis is that if someone entered into the remarketing ads campaign, they clicked an email to go to the campaign landing page. So, instead of paying a premium to serve remarketing ads to those who are only most likely to “purchase” based upon Facebook’s algorithm, but rather try to get as many people back to the donation page that saw it before, in an attempt to secure more donations overall.

Research Question

Will the “lowest cost per click” setting increase donor conversion rate for Facebook Remarketing campaigns over just using the “Purchase” optimization method?

Design

C: 2019 Board Challenge | Remarketing Campaign | Purchase
T1: 2019 Board Challenge | Remarketing Campaign | Click

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: 2019 Board Challenge | Remarketing Campaign | Purchase 0.10%
T1: 2019 Board Challenge | Remarketing Campaign | Click 0.00%-100.0% 99.5%

This experiment has a required sample size of 2,930 in order to be valid. Since the experiment had a total sample size of 19,729, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 100.0% decrease in conversion rate
× 0% increase in average gift

Key Learnings

With 99.5% level of confidence, we determined that the “lowest cost per click” remarketing target approach decreased donor conversion rates by -100%

Below outlines the other observations we made between the two campaign types, as well:

  • Impressions: The “lowest cost” (treatment) campaign increased the number of ad impressions by +54%.
  • Reach: The “purchase” optimized campaign (control) increased reach by +10%.
  • Clicks: The “lowest cost” (treatment) campaign increased the volume of clicks by +8%.
  • Cost: The “purchase” campaign was far more costly, spending +399% more of the budget when compared to the “lowest cost” (treatment) campaign.

Final Summary & Recommendation:

Although the “purchase” optimized campaign was 4 times more expensive when compared to the “lowest cost per click” setting for remarketing ad campaigns on Facebook, it was infinitely more effective at identifying people in the list that were most likely to convert, serve ads to them, then successfully bring them back to convert for the campaign.

For this reason, we would always advocate for the “purchase” optimization event to be used for remarketing campaign types, as the increased expense will generate increased results when compared to attempting to tell Facebook to spend your budget in the most frugal of ways.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #19041

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.