How an animated video ad image increased ad clickthrough rate
The Fund for American Studies
Experiment Summary
Timeframe: 09/30/2019 - 10/14/2019
As a part of optimizing The Fund for American Studies’ eBook offer, we wanted to increase ad clickthrough rate to improve traffic through the funnel. To experiment with boosting ad clickthrough, we decided to
Research Question
Will using an animated video ad from Facebook increase ad clickthrough rate?
Design
Results
This experiment has a required sample size of 75,758 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Directionally, we observed an increase of ad clickthrough rate by +7.7%, however we did not validate the experiment (achieving only an 81% level of confidence).
We turned off this experiment because after running for two weeks, we achieved the (directional) lift in ad clickthrough rate and traffic, however the email registration rate decreased (-28.5%, LoC: 100%), as did the donor conversion rate (-41.6%, LoC: 65.7%).
Our hypothesis is that although the video ad produced more traffic, with the decrease in email signups and donations, it was likely erroneous click activity after watching (or attempting to rewatch) the ad.
This approach has worked well for some offers and causes, and we should consider retesting this tactic at some point down the road.