How using a first name in the subject line impacted open rate Experiment ID: #21847

Americans for Prosperity

Experiment Summary

Timeframe: 7/18/2020 - 7/19/2020

Americans for Prosperity was looking to promote a new survey to their email house file to help identify policy alignment to formulate both segmentation opportunities, as well as understand alignment with their policy team’s perspective on the issue.

In an attempt to increase open rate (and subsequently survey fills), we decided to experiment with the subject line.

Research Question

Will adding the first name of the reader into the subject line increase open rate?

Design

C: AFP needs your voice!
T1: Cathy, make your voice heard now!

Results

Treatment Name Open Rate Relative Difference Confidence
C: AFP needs your voice! 10.3%
T1: Cathy, make your voice heard now! 11.5% 12.1% 99.8%

This experiment has a required sample size of 4,812 in order to be valid. Since the experiment had a total sample size of 23,996, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    12.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 99.8% level of confidence, we observed a +12.1% increase in open rate on the email.

Continued experimentation might be advised, as this technique may have worked on the first email when it was attempted, but over time these lifts might subside as the audience fatigues to the approach.

Nevertheless, if you’re looking to boost open rate for a single (especially a really important email) perhaps a


Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.