How a talking head video in a Facebook ad affects webinar registrations Experiment ID: #19628

NextAfter

Experiment Summary

Timeframe: 1/7/2020 - 1/20/2020

I see video-based Facebook ads all the time for different courses and trainings, and they always drive me crazy. But if they’re being used all the time, I wondered if they might just be more effective than static images. In this experiment, we tested using a static webinar title image (which has been our typical ad design for many months) against a “talking head” style video.

The video is of me in our webinar studio giving a high-level description of the webinar and what viewers can expect. It contained subtitles so that it can easily be understood with the sound off.

Research Question

Will a talking head style facebook ad lead to more webinar registrations?

Design

C: Static Image
T1: Talking Head Video

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Static Image 0.43%
T1: Talking Head Video 0.55% 28.1% 69.9%

This experiment has a required sample size of 25,001 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

When looking at results across all audiences, we saw a slight lift in conversion rate. But it did not reach a statistically significant level of confidence. However, we did find that the video ads delivered results that were 34% cheaper.

We will continue to test this in other contexts to see if the results hold up or perform better for specific audiences.


Experiment Documented by...

Nathan Hill

Nathan is an Optimization Evangelist at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.