The Gospel Coalition

How using a course trailer video to add clarity in a Facebook ad impacted click rate and cost per click

Experiment ID: #90645

The Gospel Coalition

Experiment Summary

Ended On: 04/08/2022

When reviewing performance for a recently launched online course, we were seeing a much higher cost per click for this offer when compared to other acquisition offers promoting eBook content. We wanted to find a way to reduce a cost per click and increase our engagement rate on the ads to close the gap in performance between the two offer types. We took a 2 minute video from the online course page and moved it up in the funnel to the Facebook ad level to give an introductory glimpse at what content and themes would be covered in the online course. The speaker in the video gives a clear outline of the format of each course module and how much time they can anticipate to spend which we hypothesized would help to reduce anxiety and friction in the decision making process in the mind of the donor asking “Why would I want to take this course?” and create appeal and clarity in order to drive additional interest in the offer.

Research Question

We believe that increasing clarity about the content and format of a free online course for Facebook ad viewers will achieve an increase in clicks.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Click Rate Relative Difference Confidence
C: Control 0.24%
T1: Treatment #1 0.60% 152.4% 100.0%

This experiment has a required sample size of 1,896 in order to be valid. Since the experiment had a total sample size of 135,150, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    152.4% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

We found that by switching the creative in the ad from a still image featuring the course title to a 2 minute video with clear details about what they would be signing up for, we were able to achieve a 152.4% increase in click rate on the ad with 99% level of confidence. What’s more, we were also able to reduce the cost per click by 25% so not only were we able to get more clicks but we were able to get them at a lower cost by introducing the video and giving viewers additional clarity before they took the next step to begin the registration process for the course. By reducing anxiety and friction by laying the groundwork and giving visitors clear direction in the form of an overview of what they would be signing up for, we were able to get additional people to say “yes” to take the next step and “learn more” about the course. Based on what we have learned through this experiment, we are going to move forward with testing a video trailer for an eBook offer as well to see if we can continue to improve results and be more effective with our paid media spend.


Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #90645

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.