How simplifying "Dear Reader" ads on news articles impacted donor conversion rate - NextAfter
Americans for Prosperity

How simplifying “Dear Reader” ads on news articles impacted donor conversion rate

Experiment ID: #55445

Americans for Prosperity

Experiment Summary

Timeframe: 03/24/2021 - 04/02/2021

As a part of on-site promotions of a recent open letter (petition), Americans for Prosperity wanted to experiment with simplifying their “Dear Reader” ads — which are ads that promote new offers and acquisition funnels at the end of news articles — by replicating the “simplified ad experience” that was so effective within their Facebook advertising campaign.

Would this simplified ad experience also improve results for their on-site visitors to their newsroom pages?

Research Question

We believe that simplifying the “Dear Reader” ad experience for newsroom visitors will achieve an increase in donor conversion rate.

Design

C: PRO Act Petition Promotion — Control
T1: PRO Act Petition Promotion — Simplified

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: PRO Act Petition Promotion — Control 0.17% $0.00
T1: PRO Act Petition Promotion — Simplified 0.84% 390.8% 89.6% $0.00

This experiment has a required sample size of 873 in order to be valid. Since the experiment had a total sample size of 1,185, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

With a nearly valid 89% level of confidence, we observed an increase in donor conversion rate of +390%. Because Americans for Prosperity was looking to maximize results on the limited time they had available to promote this offer, we decided to roll this out pretty quickly.

Notable changes to other metrics due to the experiment:

  • Revenue experienced a 832.5% increase with a 97% level of confidence.

Observations of the impact on other visitor segments in the experiment:

  • Email visitors had a 390.8% increase in donations with an 90% level of confidence.
  • Returning visitors had a 1.0% increase in donations with a 92% level of confidence.

It would appear that using the simplified ad experience (presenting a question, with a “Yes” or “No” button alongside it as a call to action is also as effective in on-site promotions through the “Dear Reader” ads as it was on the Facebook advertising campaign.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #55445

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.