The effect of putting the negative in all caps (subject line test)
Care Net
Experiment Summary
Ended On: 05/14/2019
This subject line test was performed in our weekly “weekend update” email.
Research Question
Does putting a negative word (“not” in this case) in all caps increase open rates?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Why abortion is not compassionate | 31.0% | ||
| T1: | Why abortion is NOT compassionate | 29.5% | -4.7% | 99.6% |
This experiment has a required sample size of 7,672 in order to be valid. Since the experiment had a total sample size of 33,201, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
4.7% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
We learned that this is actually not the case. While it didn’t make a huge difference, there was a nearly 5% decrease in email open rate.