How testing YouTube ad text affected donor acquisition costs
Hoover Institution
Experiment Summary
Ended On: 01/06/2023
At Hoover, we’ve been expanding acquisition on YouTube as we’ve seen good results thus far, and a new video became available to launch.
Our main KPI is cost per donor.
So, we wanted to try this experiment in order to identify ad copy that was more effective at driving donations and reducing acquisition costs.
We believed that testing different versions of ad copy could be a major lever in improving the performance of our YouTube ads, and that it could also help us to better understand what types of messaging resonated with our target audience.
Research Question
We believe that testing ad messaging for YouTube viewers will achieve a lower cost per donor.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.5% | ||
| T1: | Treatment #1 | 2.5% | 71.0% | 83.1% |
| T2: | Treatment #2 | 2.7% | 83.8% | 88.6% |
This experiment has a required sample size of 1,638 in order to be valid. Since the experiment had a total sample size of 2,007, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
Cost per donor
- C: $102.93
- T1: $63.07
- T2: $64.79
Testing your ad messaging:
- is easier to execute
- speed of implementation is usually faster
- can be a major lever to reducing your acquisition costs
In this case, treatment 1 saw a 39% reduction in cost per donor for 20 minutes working on ad copy.