How a video ad increased email acquisition rate Experiment ID: #16809

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Ended On: 7/2/2019

As a part of The Heritage Foundation’s acquisition efforts promoting their “What You Need to Know about Religious Freedom” ebook, we wondered what impact a video-based ad would have on our ability to acquire emails and donors.

Research Question

Would a video ad increase email registration rates?


C: Control
T1: Treatment #1


Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.77%
T1: Treatment #1 1.0% 35.7% 100.0%

This experiment has a required sample size of 8,705 in order to be valid. Since the experiment had a total sample size of 1,891,934, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 35.7% increase in conversion rate
× 0% increase in average gift

Key Learnings

With 100% level of confidence, we observed a +35.7% increase in emails acquired through the use of a video-based ad over a static image ad on the Facebook & Instagram platforms.

Because of the increased volume flowing into the donation page, we also observed the following increases, by campaign optimization type, as well:

  • Email-focused optimized campaigns: +16.9% donor conversion rate (34% LoC)
  • Purchase-focused optimized campaigns: +45.6% donor conversion rate (93% LoC)
  • Combined Total: +7.8% donor conversion rate (35% LoC)

Our hypothesis is that what is truly driving this increase in effectiveness is not anything on the either the landing page or the donation page, but rather that you are simply grabbing more people’s attention within your target audience. The motion graphics within the ad cause people to stop and pay more attention to your ad with a video-based ad.

This mental hitch that causes someone to stop to pay attention creates momentum to generate a higher click rate on video-based ads, which in turn, increases the volume of people who see the landing page.

The actual email capture rate on the landing page did not increase, it was merely a higher volume of people signing up on the landing page, because of that increased clickthrough volume from the ad platform.

Our final conclusion is to use video-based ads wherever possible to capture the higher clickthrough volumes, which should lead to increased results in terms of subscribers (or emails) acquired, as well as donors (and subsequently revenue).

Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.