Will more text on a Facebook video ad lead to more clicks to the donation page?
KUOW
Experiment Summary
Ended On: 05/01/2020
KUOW Public Radio is Seattle’s NPR news station. Their independent, nonprofit newsroom produces award-winning stories, podcasts and events. They were running a campaign and wanted to focus on securing more monthly gifts and hoped a Facebook ad could drive more people to click and go on to make a donation. They’ve seen other Facebook advertising experiments where longer copy has led to increased donations and decided to test it for themselves. In each design, there is a video that was clickable but the main call to action and hope for the ad was to get people to click Donate Now and then visit the donation page where they could continue on to complete a recurring donation.
Research Question
Will more copy lead to more clicks to the donation page?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control - Short | 0.39% | ||
| T1: | Treatment - Long | 0.41% | 5.5% | 36.6% |
This experiment has a required sample size of 662,613 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Neither the control or the treatment led to any donations and while the treatment had a slight increase in clicks, it was not significant. When looking at total unique clicks, the shorter ad (control) showed a statistically significant increase but the bulk of those were video views so, if the goal was to get more people to watch a video, it appears shorter copy is more effective. It should also be noted that the difference between the control and treatment was perhaps not great enough to prove the hypothesis. Also, taking a donor from Facebook ad to a recurring gift can be challenging.
So moving forward some ideas that require more testing and exploration are:
- Shorter ads with videos where the goal is video views or just brand awareness
- Clearly longer ads where the goal is donations or clicks to a donation page
- A short vs. long test with just an image and not a video that can be clicked
- Donation focused ads asking for a smaller one-time donation as opposed to a recurring gift