How Facebook’s optimization options can impact donor conversion
Buckner International
Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.
Experiment Summary
Timeframe: 12/22/2018 - 01/06/2019
Buckner International had been running their “7 Scriptures” offer for over a year and a half. In that time, we had run many different experiments in an effort to reduce the costs of acquiring both emails and donors. While these optimization efforts had been successful at reducing the cost per email, we had not been able to significantly impact the donor acquisition efforts.
Facebook, our primary advertising medium, had recently released a new feature called “Test and Learn.” This allows us to do a true A/B split to measure the impact of the various advertising methods offered through their platform. With this new capability, we decided to test out a differ method for optimizing our ad spent.
By default, Facebook will advertising to audiences based upon the primary conversion metric that is defined during setup. Historically, we had been optimizing our ads to find people most likely to download the offer. We decided to test optimizing for “purchases” which is a native event within the Facebook platform. A “purchase” in Buckner’s case was a donation. We had learned that Facebook incorporates outside purchasing information when using their native event so, despite having much smaller conversion number, we wanted to see if Facebook would be able to successfully model a likely donor for the organization using this outside data.
Research Question
Will optimizing our ads to find likely “purchasers” allow us to acquire more donors at a better cost?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Ad Optimization for Emails | 0.01% | ||
| T1: | Ad Optimization for "Purchases" | 0.05% | 357.7% | 99.3% |
This experiment has a required sample size of 14,934 in order to be valid. Since the experiment had a total sample size of 68,852, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 357.7% increase in conversion rate
× 0% increase in average gift
Key Learnings
Utilizing Facebook’s ad optimization to find likely donors, we were able to acquire 357% more donors for the same ad spend. We went from a 0.3% instant donor conversion rate to a 2.4% conversion rate.
Now, this method of optimization had the negative effect of increasing our cost per email by 115% ($2.67 per email for purchase optimizations vs. $1.24 per email for email optimization). However, since the primary goal of our email acquisition efforts is to acquire a donor both immediately and downstream, this was an easy tradeoff to make.
Future experimentation will focus on additional ad optimization features, like targeting specific cost per conversion metrics, and running more experiments on the landing page to reduce our cost per email.