How using a first name in the subject line impacted open rate
Americans for Prosperity
Experiment Summary
Timeframe: 07/18/2020 - 07/19/2020
Americans for Prosperity was looking to promote a new survey to their email house file to help identify policy alignment to formulate both segmentation opportunities, as well as understand alignment with their policy team’s perspective on the issue.
In an attempt to increase open rate (and subsequently survey fills), we decided to experiment with the subject line.
Research Question
Will adding the first name of the reader into the subject line increase open rate?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | AFP needs your voice! | 10.3% | ||
| T1: | Cathy, make your voice heard now! | 11.5% | 12.1% | 99.8% |
This experiment has a required sample size of 4,812 in order to be valid. Since the experiment had a total sample size of 23,996, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
12.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
With a 99.8% level of confidence, we observed a +12.1% increase in open rate on the email.
Continued experimentation might be advised, as this technique may have worked on the first email when it was attempted, but over time these lifts might subside as the audience fatigues to the approach.
Nevertheless, if you’re looking to boost open rate for a single (especially a really important email) perhaps a