How a talking head video in a Facebook ad affects webinar registrations Experiment ID: #19603

NextAfter

Experiment Summary

Timeframe: 1/7/2020 - 1/20/2020

I see video-based Facebook ads all the time for different courses and trainings, and they always drive me crazy. But if they’re being used all the time, I wondered if they might just be more effective than static images. In this experiment, we tested using a static webinar title image (which has been our typical ad design for many months) against a “talking head” style video.

The video is of me in our webinar studio giving a high-level description of the webinar and what viewers can expect. It contained subtitles so that it can easily be understood with the sound off.

Research Question

Will a talking head style facebook ad lead to more webinar registrations?

Design

C: Static
T1: Talking Head

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Static 0.58%
T1: Talking Head 1.4% 136.8% 96.6%

This experiment has a required sample size of 1,187 in order to be valid. Since the experiment had a total sample size of 2,778, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 136.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

Looking specifically at our “All Site Traffic” retargeting audience, we saw a significant, 136.8% increase in webinar registrations. On top of that, our cost per registrant was 54% lower. We will continue to test more video-based ads that use the personal and organic approach to see if the results hold true and apply in other contexts.


Experiment Documented by...

Nathan Hill

Nathan is an Optimization Evangelist at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.