How new Facebook ad creative impacted donor conversion - NextAfter
Dallas Theological Seminary

How new Facebook ad creative impacted donor conversion

Experiment ID: #69006

Dallas Theological Seminary

The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.

Experiment Summary

Ended On: 08/09/2021

As we significantly increased media budget on Facebook ads, we wanted to run treatment versions of existing free online course ads to optimize our spend and avoid creative fatigue for the digital audiences we were targeting. For the first treatment, we developed new copy that was less academic in nature and introduced experiential language that reflected more of the audience identity. We also increased clarity in the offer by outlining what someone could expect: “Through this course, you’ll find encouragement in God’s Word to deal with daily issues with an eternal mindset.” For the second treatment, we kept the copy the same as the control and simply used a graphic of a Bible verse relevant to the course topic to demonstrate alignment of values and identity.

Research Question

We believe that aligning identity through copy and graphics for Facebook ad audiences will achieve improved ad performance.

Design

C: Control
T1: Treatment #1
T2: Treatment #2

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.01%
T1: Treatment #1 0.01% 32.4% 45.9%
T2: Treatment #2 0.02% 227.7% 99.5%

This experiment has a required sample size of 75,620 in order to be valid. Since the experiment had a total sample size of 558,376, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 227.7% increase in conversion rate
× 0% increase in average gift

Key Learnings

What we found after 3 weeks of running all 3 versions simultaneously was the verse image treatment saw a 227% increase in donor version with a 99.4% level of confidence. It also drove the click rate up from .7% on the control to 1.3% with 100% level of confidence. The treatment that had copy changes only did not validate for donor conversion, and actually decreased the click rate to .5% with 100% level of confidence. The next iteration of the test will be to test a version of the ad with the treatment copy paired with the treatment image so that the concept of identity alignment is present in both elements. We are also testing similar graphics on other online course ads to see if these results are consistent across all offers.


Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #69006

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.