How condensed and intriguing hyperlink copy impacted newsletter clickthrough rates
Alley Cat Allies
Experiment Summary
Ended On: 07/24/2020
As a part of iterative testing with the Alley Cat Allies newsletter in the name of increasing email engagement rates, we wanted to experiment with using shorter, more intriguing article hyperlinks within the monthly newsletter to determine what impact it might have moving forward.
Research Question
Will shorter and more intriguing link descriptions increase newsletter clickthrough rate?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.92% | ||
| T1: | Article Teaser in Hyperlinks | 1.0% | 13.2% | 96.3% |
This experiment has a required sample size of 50,892 in order to be valid. Since the experiment had a total sample size of 115,028, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
13.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
With a 96.2% level of confidence, we observed a +13.2% increase in clickthrough rate from the treatment.
The basic thesis of this experiment is that by giving away less of what happens “post-click” away from the articles being presented in the newsletter, and instead by creating “intrigue” for what happens after the reader clicks to learn more, we can drive better newsletter clickthrough rates moving forward.
Not only does this approach generate more engagement, but it is also an easier version of the monthly newsletter to create. This means that the team can spend more time focusing upon writing teasers for what is in the article and move on with other priorities to drive better newsletter engagement moving forward.