Hoover Institution

How segmenting audiences vs donor audience signal affected donor conversion rate on YouTube

Experiment ID: #122542

Hoover Institution

Experiment Summary

Ended On: 01/06/2023

Seeing YouTube as the next opportunity of paid media channel expansion, we wanted to see the difference in performance between giving Google all of the data and allowing the algorithm to optimize the audience versus segmenting our audiences by stages of awareness.

Research Question

We believe that segmenting audiences for YouTube will achieve a higher donor conversion rate.


C: Control
T1: TOF - Video Viewers
T2: MOF – FB Web Visitors
T3: BOF - Donors
T4: Similar - Donors


 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.18%
T1: TOF - Video Viewers 0.90%413.9% 100.0%
T2: MOF – FB Web Visitors 1.4%725.5% 100.0%
T3: BOF - Donors 5.6%3,076.6% 100.0%
T4: Similar - Donors 1.2%583.3% 100.0%

This experiment has a required sample size of 74 in order to be valid. Since the experiment had a total sample size of 19,331, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 413.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

This experiment highlights the importance of segmenting audiences in order to achieve the best possible results from YouTube advertising.

This showed that segmenting audiences based on factors such as YouTube video views, Facebook web visits, and donor status resulted in significantly higher conversion rates compared to the control.

In particular, the BOF – Donors group saw a conversion rate that was more than 30 times higher than the control group, and the MOF – FB Web Visitors group saw a conversion rate that was more than 7 times higher. These results suggest that by targeting specific groups of users, it is possible to achieve much higher conversion rates and to more effectively drive donations. 

Other Metrics

Cost per donor:

  • C: $315.53 (0.14roi)
  • T1: $171.74 (0.67roi)
  • T2: $130.82 (0.35roi)
  • T3: $85.84 (4.47roi)
  • T4: $163.78 (0.23roi)

Cost per email:

  • C: $6.29 (502)
  • T1: $2.90 (1898)
  • T2: $3.75 (1467)
  • T3: $7.18 (765)
  • T4: $4.95 (3021)

Segmenting your audiences for YouTube ads can significantly improve the donor conversion rate.

Experiment Documented by Riley Young
Riley Landenberger is Audience Engagement Manager at NextAfter.

Question about experiment #122542

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.