Impact of including the CTA in the email subject line
Care Net
Experiment Summary
Ended On: 10/27/2020
For our bi-monthly email newsletter, I wanted to test including the call to action in the subject line. These newsletters are always sent from our CEO’s email address and I wanted to see if him saying the reader should do something would increase their likelihood of engaging with the content. For both versions, I used the same preview text which remains unchanged from newsletter to newsletter.
Research Question
Will having the CTA in the subject line increase conversion rate?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | You helped this single mom choose life | 17.8% | ||
| T1: | Something you should read | 21.0% | 18.2% | 100.0% |
This experiment has a required sample size of 1,147 in order to be valid. Since the experiment had a total sample size of 62,946, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
18.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
Having the conversational CTA in the subject line increased not only the open rate, but the click through rate as well. The CTR increased 41%! People were primed from the subject line to click the articles in the Newsletter to read them on the site. This is something we will continue to test in future emails, especially when there is something important we want the reader to do.