How impression-based advertising to donor audiences impacted conversion
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 08/20/2021 - 09/01/2021
DTS had been promoting their online course, Jonah, via Facebook to several different audience groups. One of those segments was a set of modeled audiences from Wiland. This included the specifically modeled donors and donor lookalikes generated on Facebook based upon the Wiland segments. We had seen declining results from these groups so we wanted to test different tactics to get more out of these prime audiences.
It had been suggested to us that since these were identified donor prospects, we should change our advertising optimization metrics from “Purchases” (aka donations) to be based upon “Reach”. The hypothesis was that showing the offer to more prospective donors would reduce ad costs while improving results. We decided to test this hypothesis against the standard Purchase optimization.
Research Question
We believe that optimizing for impressions for Wiland-modeled audiences will achieve improved cost per donor results.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Ad Objective: Purchases | 0.03% | ||
| T1: | Ad Objective: Reach | 0.00% | -100.0% | 100.0% |
This experiment has a required sample size of 3,965 in order to be valid. Since the experiment had a total sample size of 717,614, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 100.0% decrease in conversion rate
× 0% increase in average gift
Key Learnings
After spending $2,500 on each of the two campaigns, we found that the Purchase optimized campaign resulted in 26 new donors while the Reach optimized campaign received zero gifts. This is similar to the kind of results we see with donor lookalike audiences generated by Facebook when we use “Reach” as the optimization metrics.
It should be noted that both the “Reach” and “Purchase” campaigns spent a greater proportion of budget on the Gold and Platinum lookalike audiences. However, the native Wiland audiences received several hundred in spend each while returning zero donors for the Reach campaign.