Hoover Institution

How testing YouTube ad text affected donor acquisition costs

Experiment ID: #122540

Hoover Institution

Experiment Summary

Ended On: 01/06/2023

At Hoover, we’ve been expanding acquisition on YouTube as we’ve seen good results thus far, and a new video became available to launch.

Our main KPI is cost per donor.

So, we wanted to try this experiment in order to identify ad copy that was more effective at driving donations and reducing acquisition costs.

We believed that testing different versions of ad copy could be a major lever in improving the performance of our YouTube ads, and that it could also help us to better understand what types of messaging resonated with our target audience.

Research Question

We believe that testing ad messaging for YouTube viewers will achieve a lower cost per donor.


C: Control
T1: Treatment #1
T2: Treatment #2


 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 1.5%
T1: Treatment #1 2.5%71.0% 83.1%
T2: Treatment #2 2.7%83.8% 88.6%

This experiment has a required sample size of 1,638 in order to be valid. Since the experiment had a total sample size of 2,007, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

Cost per donor

  1. C: $102.93
  2. T1: $63.07 
  3. T2: $64.79 

Testing your ad messaging:

  • is easier to execute
  • speed of implementation is usually faster
  • can be a major lever to reducing your acquisition costs

In this case, treatment 1 saw a 39% reduction in cost per donor for 20 minutes working on ad copy.

Experiment Documented by Riley Young
Riley Landenberger is Audience Engagement Manager at NextAfter.

Question about experiment #122540

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.