How simplifying “Dear Reader” ads on news articles impacted donor conversion rate
Americans for Prosperity
Experiment Summary
Timeframe: 03/24/2021 - 04/02/2021
As a part of on-site promotions of a recent open letter (petition), Americans for Prosperity wanted to experiment with simplifying their “Dear Reader” ads — which are ads that promote new offers and acquisition funnels at the end of news articles — by replicating the “simplified ad experience” that was so effective within their Facebook advertising campaign.
Would this simplified ad experience also improve results for their on-site visitors to their newsroom pages?
Research Question
We believe that simplifying the “Dear Reader” ad experience for newsroom visitors will achieve an increase in donor conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | PRO Act Petition Promotion — Control | 0.17% | $0.00 | ||
| T1: | PRO Act Petition Promotion — Simplified | 0.84% | 390.8% | 89.6% | $0.00 |
This experiment has a required sample size of 873 in order to be valid. Since the experiment had a total sample size of 1,185, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
With a nearly valid 89% level of confidence, we observed an increase in donor conversion rate of +390%. Because Americans for Prosperity was looking to maximize results on the limited time they had available to promote this offer, we decided to roll this out pretty quickly.
Notable changes to other metrics due to the experiment:
- Revenue experienced a 832.5% increase with a 97% level of confidence.
Observations of the impact on other visitor segments in the experiment:
- Email visitors had a 390.8% increase in donations with an 90% level of confidence.
- Returning visitors had a 1.0% increase in donations with a 92% level of confidence.
It would appear that using the simplified ad experience (presenting a question, with a “Yes” or “No” button alongside it as a call to action is also as effective in on-site promotions through the “Dear Reader” ads as it was on the Facebook advertising campaign.