How promoting a petition in the "Dear Reader" call to action promotion on news articles impacted email signup rates - NextAfter
Americans for Prosperity

How promoting a petition in the “Dear Reader” call to action promotion on news articles impacted email signup rates

Experiment ID: #81991

Americans for Prosperity

Experiment Summary

Timeframe: 01/12/2022 - 01/24/2022

As a part of experimenting with the promotion of newsroom articles, Americans for Prosperity wanted to test whether promoting a news-relevant petition in their “Dear Reader” promotion (which is a call to action at the end of a news article on their website) would be more productive than promoting their email subscription offer.

Research Question

We believe that promoting a relevant petition for article readers will achieve increased email signup rates.

Design

C: Subscription Promotion
T1: Relevant Petition

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Subscription Promotion 6.8% $0.00
T1: Relevant Petition 20.0% 196.1% 100.0% $0.00

This experiment has a required sample size of 51 in order to be valid. Since the experiment had a total sample size of 2,532, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 196.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 100% level of confidence, we observed nearly a 3x increase in email signup rates (+196.1%) when promoting the news-relevant petition.

What’s also interesting to note is that the treatment petition offer also offers access to the subscription type offered in the control ad, but it’s the optional checkbox shown before the button to submit your petition form.

The data shows that even though this box is unchecked by default — meaning that if someone wanted to sign up for that subscription type, they would have to go through an extra step of checking the box before submitting the form — the treatment saw that 90.7% of the people who signed the treatment form also checked that box.

This means that the treatment with its unchecked, optional subscription offer produced +58.8% more subscriptions to the primary offer made in the control experience (confidence level: 100%).

This means that we should look to promote petitions on the news articles moving forward. The next challenge will be to understand/determine how this offer performs when the news cycle shifts away from the offer in question.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #81991

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.