How including a first name personalization in the subject line of a recurring donor cultivation email affects open rates
KCBI
Experiment Summary
Timeframe: 02/11/2021 - 02/11/2021
KCBI was sending out a cultivation emails to their recurring donors, letting them know all of the ways their recurring gifts were going to change lives this year. We wanted to test a personalized subject line that included the donor name to see how that affected the open rate, because one of our goals is to increase cultivation for their monthly donors to continue to increase their already excellent recurring donor retention rate. We split the email by active and inactive emails (hadn’t opened the last five emails), so that we could see the affects on engaged audience versus unengaged.
Research Question
Personalizing the subject line for a more one-to-one communication will increase open rates.
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 5.2% | ||
| T1: | Treatment #1 | 8.0% | 54.8% | 97.9% |
This experiment has a required sample size of 589 in order to be valid. Since the experiment had a total sample size of 1,622, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
54.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
In the inactive segment, the personalization made a big impact in opens, increasing the number of folks opening by 54%. Our hypothesis that personalization would increase email reactivation through a more one-to-one communication proved to be true.
The active segment only saw a small increase in open rate, not enough to be statistically valid. We believe this is because this is an already very engaged audience, as recurring donors who regularly open emails.