How including a first name personalization in the subject line of a recurring donor cultivation email affects open rates

Experiment ID: #51274


Experiment Summary

Timeframe: 02/11/2021 - 02/11/2021

KCBI was sending out a cultivation emails to their recurring donors, letting them know all of the ways their recurring gifts were going to change lives this year. We wanted to test a personalized subject line that included the donor name to see how that affected the open rate, because one of our goals is to increase cultivation for their monthly donors to continue to increase their already excellent recurring donor retention rate. We split the email by active and inactive emails (hadn’t opened the last five emails), so that we could see the affects on engaged audience versus unengaged.

Research Question

Personalizing the subject line for a more one-to-one communication will increase open rates.


C: Control
T1: Treatment #1


 Treatment NameOpen RateRelative DifferenceConfidence
C: Control 5.2%
T1: Treatment #1 8.0%54.8% 97.9%

This experiment has a required sample size of 589 in order to be valid. Since the experiment had a total sample size of 1,622, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    54.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

In the inactive segment, the personalization made a big impact in opens, increasing the number of folks opening by 54%. Our hypothesis that personalization would increase email reactivation through a more one-to-one communication proved to be true.

The active segment only saw a small increase in open rate, not enough to be statistically valid. We believe this is because this is an already very engaged audience, as recurring donors who regularly open emails.

Experiment Documented by NextAfter

Question about experiment #51274

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.