How the type of subject line affects open rate in an appeal Experiment ID: #8483
National Breast Cancer Foundation
The National Breast Cancer Foundation's mission is to provide help and inspire hope to those affected by breast cancer through early detection, education, and support services.
Ended On: 1/24/2018
NBCF was sending out an email appeal to their housefile for their Valentine’s Day Hope Kit campaign. Their goal for the experiment was to get more eyes on the email and more opens. The hypothesis was that the subject line “720 women” would be more compelling than asking the reader a question or to do something, for example, “Will you help me do this?”. The email itself remained the same
Does the type of subject in an email affect open rate?
|Treatment Name||Open Rate||Relative Difference||Confidence|
|C:||Will you help me share some love?||13.7%|
This experiment has a required sample size of 1,418 in order to be valid. Since the experiment had a total sample size of 23,070, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
19.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
The treatment headline produced a 19% increase in open rate of the email appeal. This is possibly due to the workload involved in the first subject line, which requires “work” on the reader’s behalf and immediately adds friction to the opening process, and partially due to the compelling nature of the treatment, “720 women”, which refers to the 720 women diagnosed with breast cancer every day.