How video ads affect email acquisition rate for a free course
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Timeframe: 02/03/2017 - 03/03/2017
DTS was promoting their new course, called Can You Trust the Bible?. They had a special video from their Academic Dean to promote it and wanted to test using this video to promote the course. Video ads had shown in some experiments to improve conversion for offers, although DTS had previously used static ads.
They split audiences and tested each ad unit to see which resulted in a better cost-per-acquisition.
Will video ads produce an increase in email acquisition rate?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 420 in order to be valid. Since the experiment had a total sample size of 1,633, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 37.1% decrease in conversion rate
× 0% increase in average gift
The video ads produced a 37% decrease in conversion when tested against the static ads. While a deeper look at the data showed that many more people engaged with the video ads (which means they clicked somewhere on the ad), the video ad was either too distracting or not compelling enough to convince them to take the next step and sign up for the course. This seems to indicate that, in this case, video is more of a distraction than a motivator to take the course.
This doesn’t entirely rule out video ads on Facebook. There have been experiments where video ads produce a lift, and they can be effective at remarketing. DTS will continue to test video as a promotional tool for their free courses.
Question about experiment #6239
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.