How SpotRight compares to other Facebook Audience Models for prospecting
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Ended On: 03/05/2020
The Heritage Foundation wanted to experiment with using a modeled audience from the new SpotRight tool available from Wiland, one of their data partners. SpotRight claims to use a proprietary model to produce a higher prospective customer list that is 10x more likely to become a buyer.
In an attempt to validate the audiences it built, we setup an experiment with a Heritage modeled audience based upon targeting keywords and pushed approximately 20 million people into a targetable audience on Facebook. We then gave the campaign a modest budget (about 5% for the quarter) and a 19-day timeline to run our existing prospecting ads.
Research Question
How does SpotRight’s audience compare to other audiences used in prospecting and targeting for new donor acquisition?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.01% | ||
| T1: | Treatment #1 | 0.01% | 157.5% | 99.9% |
| T2: | Treatment #2 | 0.01% | 141.0% | 99.0% |
| T3: | Treatment #3 | 0.01% | 35.5% | 70.0% |
This experiment has a required sample size of 164,816 in order to be valid. Since the experiment had a total sample size of 1,342,906, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 157.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
In reviewing the results from the targeting experiment, we can conclusively say that the SpotRight modeled audience increased donor conversion rate over the “control” (which is targeting fans of similar causes and/or interest-based audiences) by 141%.
We also noticed that the SpotRight modeled audience performed as well as the Facebook lookalike modeled audiences. What does this mean? It means that SpotRight may be a good starting point when launching a new brand, program or fundraising initiative if you do not have the minimum number of donors to model an audience from.
Because Facebook audiences should really be thought of as “overlapping circles on a Venn Diagram” that often overlap when you build multiple audiences that you are targeting, we thought that many of our known audiences (e.g. donors and/or subscribers) may have found their way into the SpotRight audience model.
So, we checked the Analytics on the visitors, and we found that 85% of all SpotRight visitors to our donation pages actually were first time visitors to our website, and when they converted, their average gift was only slightly less (-7%) than return visitors that came through the SpotRight campaign.
Where do we go from here? We wonder how this audience can scale moving forward, so we’ve quadrupled the budget to monitor how the SpotRight audience scales over a longer period of time (31 days) with more budget from the original experiment time frame.