How target ROAS targeting on Facebook campaigns decreased email acquisition rates
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Ended On: 03/31/2019
In an attempt to boost immediate ROI on The Heritage Foundation’s Guide to the Constitution offer, we implemented a Target ROAS campaign setting within the Facebook Test & Learn platform.
Research Question
What impact does Target ROAS have upon the overall acquisition funnel for The Heritage Guide to the Constitution offer?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.75% | ||
| T1: | Treatment #1 | 0.41% | -45.1% | 100.0% |
This experiment has a required sample size of 4,088 in order to be valid. Since the experiment had a total sample size of 376,867, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 45.1% decrease in conversion rate
× 0% increase in average gift
Key Learnings
With a 100% level of confidence, we were able to identify a 45.1% decrease in emails acquired with a Target ROAS setting within Facebook campaign ad targeting.
We will continue to monitor for further learnings about introducing this campaign targeting type alongside the “Lowest Cost” setting, but the other learnings (that have not yet been validated) are as follows:
- Target ROAS also increased instant donor conversion rate by +32.7% (~ 70% valid)
- Target ROAS also improved average gift values by +11.5%
- Target ROAS also increase revenue/visitor by +38.6% (~ 75% valid)
The biggest problem we’ve noticed besides the decreased email conversion rate (shown within this experiment) is that we are having tremendous issues with scaling costs and reach.
The Target ROAS campaign has spent -60% less than the “Lowest Cost” (control) target campaign type. For this reason, we actually are continuing to run this experiment alongside the “Lowest Cost” campaign, because we’re achieving the “volume” of donors and dollar that we need within the program, but by leaving the Target ROAS campaign running alongside it, we have brought up the efficiency we need within the overall campaign expense on this offer.
Finally, our last observation with this experiment is related to the $100 premium gift offer that is included in the page.
A total of 61% of the donors that the “Target ROAS” (treatment) campaign produced converted at/above the $100 premium gift offer, where the “Lowest Cost” (control) campaign converted donors at/above the $100 at a 53.6% rate. This means that the “Target ROAS” (treatment) type of campaign produced increased the “premium donor” conversion rate by +15%.
Moving forward, it might be advisable to launch with a Target ROAS component to your campaign if any/all of the conditions below exist:
- You have limited budget.
- You have a long-running campaign but want to improve efficiency/ROI metrics.
- You don’t care about reach/volume, but only care about controlling costs in your acquisition efforts.