How condensed and intriguing hyperlink copy impacted newsletter clickthrough rates – NextAfter
Alley Cat Allies

How condensed and intriguing hyperlink copy impacted newsletter clickthrough rates

Experiment ID: #21354

Alley Cat Allies

Experiment Summary

Ended On: 7/24/2020

As a part of iterative testing with the Alley Cat Allies newsletter in the name of increasing email engagement rates, we wanted to experiment with using shorter, more intriguing article hyperlinks within the monthly newsletter to determine what impact it might have moving forward.

Research Question

Will shorter and more intriguing link descriptions increase newsletter clickthrough rate?


C: Control
T1: Article Teaser in Hyperlinks


Treatment Name Click Rate Relative Difference Confidence
C: Control 0.92%
T1: Article Teaser in Hyperlinks 1.0% 13.2% 96.3%

This experiment has a required sample size of 50,892 in order to be valid. Since the experiment had a total sample size of 115,028, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    13.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 96.2% level of confidence, we observed a +13.2% increase in clickthrough rate from the treatment.

The basic thesis of this experiment is that by giving away less of what happens “experiments-click” away from the articles being presented in the newsletter, and instead by creating “intrigue” for what happens after the reader clicks to learn more, we can drive better newsletter clickthrough rates moving forward.

Not only does this approach generate more engagement, but it is also an easier version of the monthly newsletter to create. This means that the team can spend more time focusing upon writing teasers for what is in the article and move on with other priorities to drive better newsletter engagement moving forward.

Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #6579

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.