Impact of a personal subject line on a calendar year end appeal
Care Net
Experiment Summary
Timeframe: 11/28/2018 - 12/04/2018
We wanted to see which subject line would perform better during a calendar year end campaign. One that had an element of mystery and alarm or one that was very personal and short, like something a friend would send.
Research Question
Does a personal and short subject line perform better on calendar year end appeals?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Your money may be funding abortion this Christmas | 8.4% | ||
| T1: | Are you busy? | 13.3% | 58.8% | 100.0% |
This experiment has a required sample size of 306 in order to be valid. Since the experiment had a total sample size of 82,173, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
58.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The short, personal subject line felt like something a friend or family member would send while the mystery subject line sounded more institutional in nature. As a result, the short performed much better.