NextAfter

How previewing the key takeaways of a blog post in an email vs. shorter copy to share the link

Experiment ID: #64305

NextAfter

Experiment Summary

Timeframe: 05/07/2021 - 05/07/2021

In a NIO cultivation email where we were sharing a blog post, we wondered if it would be more effective to preview some key takeaways from the post or simply just share the link with short copy. There are 2 questions here – will a preview of key takeaways make the content seem more valuable? OR will shorter copy create curiosity to read the post?

Research Question

We believe that shortening the copy and simply sharing a link for participants in our NIO cultivation experiment will achieve an increase in clicks to the full post.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Click Rate Relative Difference Confidence
C: Control 1.8%
T1: Treatment #1 3.7% 105.0% 100.0%

This experiment has a required sample size of 573 in order to be valid. Since the experiment had a total sample size of 8,316, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    105.0% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

We learned in this experiment that simply sharing the link rather than preview the blog post is much more successful in the click rate to the full post. We can assume that because the control email shared key takeaways and imagery, it gave too much away and made it less necessary to visit the full post.


Experiment Documented by NextAfter

Question about experiment #64305

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.