How previewing the key takeaways of a blog post in an email vs. shorter copy to share the link
NextAfter
Experiment Summary
Timeframe: 05/07/2021 - 05/07/2021
In a NIO cultivation email where we were sharing a blog post, we wondered if it would be more effective to preview some key takeaways from the post or simply just share the link with short copy. There are 2 questions here – will a preview of key takeaways make the content seem more valuable? OR will shorter copy create curiosity to read the post?
Research Question
We believe that shortening the copy and simply sharing a link for participants in our NIO cultivation experiment will achieve an increase in clicks to the full post.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.8% | ||
| T1: | Treatment #1 | 3.7% | 105.0% | 100.0% |
This experiment has a required sample size of 573 in order to be valid. Since the experiment had a total sample size of 8,316, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
105.0% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
We learned in this experiment that simply sharing the link rather than preview the blog post is much more successful in the click rate to the full post. We can assume that because the control email shared key takeaways and imagery, it gave too much away and made it less necessary to visit the full post.