Care Net

The impact of using Facebook’s video creation tool on an ad campaign

Experiment ID: #18592

Care Net

Experiment Summary

Timeframe: 07/11/2019 - 08/11/2019

We’ve had a successful campaign advertising a pledge that led to a monthly donation page on Facebook. In a previous experiment, we found that using an image of a newborn baby on this offer significantly increased donor conversion (https://app.winstonknows.com/experiment/2321)   We decided to test that image against a new video using the video creation tool in Facebook. They hypothesis was that a 15sec video would get more people to convert than the image. 

Research Question

Will using a 15sec video created with the Facebook video creation tool convert more people into donors on the ad than a photo?

Design

C: Infanticide 365 photo ad
T1: Infanticide 365 Video Ad

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Infanticide 365 photo ad 0.17%
T1: Infanticide 365 Video Ad 0.55%218.5% 94.9%

This experiment has a required sample size of 1,818 in order to be valid. Since the experiment had a total sample size of 4,127, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

218% more people who clicked the video ad became monthly donors than who clicked the photo ad. This was despite the fact that the photo ad had  14% more impressions. We will begin using and testing videos in more of our campaigns to see the impact they have on conversion. 


Experiment Documented by Ardee Coolidge

Question about experiment #18592

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.