How using an animated video for an ad image impacted donor conversion rate Experiment ID: #21568

The Fund for American Studies

Experiment Summary

Ended On: 7/1/2020

As a part of launching a new eBook offer, The Fund for American Studies (TFAS) was looking for a way to increase ad clickthrough and subsequently donor conversion and revenue from the launch campaign. They wondered if using the pre-fabricated image-based animations to create a video ad would increase results.

Research Question

Will an animation-based ad image (or rather, a video ad) increase donor conversion rate or revenue?

Design

C: Static Image
T1: Animated Video

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Static Image 0.03%
T1: Animated Video 0.09% 187.6% 98.3%

This experiment has a required sample size of 14,808 in order to be valid. Since the experiment had a total sample size of 38,791, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 187.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 98.3% level of confidence, we observed a +187.6% increase in donor conversion rate when using a video ad when compared to the static image ad type.

Subsequently, this increase in donor conversion also increased revenue by +151.6% (LoC: 98.3%).

Ironically, we did not see a valid increase in ad clickthrough rate from this approach (only achieving a +7.1%, with a 73% level of confidence).

Our hypothesis is that the video ad holds people for a moment longer, catching their attention, which then engages them in the ad copy. With greater ad copy consumption, they arrive at the landing page better motivated to get the eBook, at which point they’re arriving at the donation page with a higher motivation to convert into a donor, as well.

Further testing may be required, but if you’re looking to increase your down-funnel conversion metrics, simply experimenting with a video-based ad from the wizard within Facebook’s Ad Manager platform could yield a more motivated audience, which could increase your results, as well.


Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.