How a positive subject line affects open rates
Media Research Center
The Media Research Center (MRC) is the nation’s premier media watchdog. The sole mission is to expose and neutralize the propaganda arm of the Left: the national news media.
Experiment Summary
Timeframe: 09/24/2015 - 09/26/2015
As part of an effort to increase engagement in their moneybomb campaign, the Media Research Center wanted to discover what messaging tone would best engage their audience. The idea was to send two different subject lines to 20% of their audience and then send the best performing line to the remaining 80%.
The previous emails had utilized a negative tone in the subject line as a way of introducing the “threat” associated with the campaign. We had the hypothesis that a positive tone would actually resonate more with constituents so we decided to test it.
Research Question
Does a positive subject line increase the constituent’s engagement with the email over a negative subject line?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Concerned about the future | 11.0% | ||
| T1: | I have hope | 13.0% | 17.9% | 100.0% |
This experiment has a required sample size of 2,099 in order to be valid. Since the experiment had a total sample size of 64,240, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
17.9% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The positive subject line increased the open rate by 17.9% compared to the more negative tone. This increased open rate led to a 26.5% increase to the number of visits to the website.
This test allowed us to increase the overall open rate of the email while teaching us an important fact about the email file. The “fear of loss” can be a powerful motivator but there are times where a more inspirational message can be even more motivating.