How a pledge at the end of an article impacts email signup rate Experiment ID: #28790

Leadership Institute

Experiment Summary

Ended On: 8/21/2020

The Leadership Institute runs a daily news website called Campus Reform, where leaders can read articles that covers what is happening on college campuses related to free speech and education.

With two subscription types (daily and weekly), there are a substantial number of visitors to this website each year.

Readers are presented with opportunities to become a subscriber through a sticky bar at the foot of each article page that pops up when a visitor arrives on the website.

But could we increase email subscribers by trying a different location?

Research Question

Would adding a pledge email signup offer at the end of the article on the Campus Reform website increase email registration rate?

Design

C: Stickybar
T1: End of Article Pledge

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Stickybar 0.12%
T1: End of Article Pledge 0.61% 396.2% 100.0%

This experiment has a required sample size of 1,180 in order to be valid. Since the experiment had a total sample size of 128,145, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 396.2% increase in conversion rate
× 0% increase in average gift

Key Learnings

With 100% level of confidence, we observed a +396.2% increase in email registration rate with the pledge.

This end of article pledge was designed to create donor momentum towards the subsequent recurring donor appeal, however, we observed a massive increase in motivating people to sign the pledge, subsequently opting them in to receive the daily news email, as well.


Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.