Impact of First Name personalization in the subject line
Care Net
Experiment Summary
Timeframe: 10/30/2021 - 11/03/2021
We were wondering if casually and conversationally using {first name} personalization in the subject line would increase the email open rate for non-donors.
Research Question
We believe that using personalization in the subject line for our email subscribers will achieve more opens.
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Have you heard of him? | 36.1% | ||
| T1: | Candice, have you heard of him? | 40.7% | 12.6% | 100.0% |
This experiment has a required sample size of 881 in order to be valid. Since the experiment had a total sample size of 65,117, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
12.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
Adding the first name increased our open rate by 12%. We will continue to look for ways to conversationally and authentically use a recipient’s first name in the subject line to increase their open rate.
Notable changes to other metrics due to the experiment:
- Clicks to Opens experienced a 12.5% decrease with a 97% level of confidence.