How new Facebook ad creative impacted donor conversion
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Ended On: 08/09/2021
As we significantly increased media budget on Facebook ads, we wanted to run treatment versions of existing free online course ads to optimize our spend and avoid creative fatigue for the digital audiences we were targeting. For the first treatment, we developed new copy that was less academic in nature and introduced experiential language that reflected more of the audience identity. We also increased clarity in the offer by outlining what someone could expect: “Through this course, you’ll find encouragement in God’s Word to deal with daily issues with an eternal mindset.” For the second treatment, we kept the copy the same as the control and simply used a graphic of a Bible verse relevant to the course topic to demonstrate alignment of values and identity.
Research Question
We believe that aligning identity through copy and graphics for Facebook ad audiences will achieve improved ad performance.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.01% | ||
| T1: | Treatment #1 | 0.01% | 32.4% | 45.9% |
| T2: | Treatment #2 | 0.02% | 227.7% | 99.5% |
This experiment has a required sample size of 75,620 in order to be valid. Since the experiment had a total sample size of 558,376, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 227.7% increase in conversion rate
× 0% increase in average gift
Key Learnings
What we found after 3 weeks of running all 3 versions simultaneously was the verse image treatment saw a 227% increase in donor version with a 99.4% level of confidence. It also drove the click rate up from .7% on the control to 1.3% with 100% level of confidence. The treatment that had copy changes only did not validate for donor conversion, and actually decreased the click rate to .5% with 100% level of confidence. The next iteration of the test will be to test a version of the ad with the treatment copy paired with the treatment image so that the concept of identity alignment is present in both elements. We are also testing similar graphics on other online course ads to see if these results are consistent across all offers.