How a personalizing the “envelope” of an email affects open rate
Compassion International
Experiment Summary
Ended On: 10/24/2019
Compassion International invited young college graduates to join their Compassion Go internship program through an email offer. The email came from “Compassion International” as the sender, and they wanted to know if personalizing the “envelope” on the email could increase open rate. They changed the sender to the name of an intern, and changed the subject line from a very sales-heavy subject line to “Quick question”. Then, they ran a test to see which one produced a higher open rate.
Research Question
How will personalizing the “envelope” of an email affect open rate?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 12.0% | ||
| T1: | Personal Style | 51.4% | 329.8% | 100.0% |
This experiment has a required sample size of 11 in order to be valid. Since the experiment had a total sample size of 50,668, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
329.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment email had a massive 329% increase in open rate. However, this “personal style” wasn’t just applied to the subject line and sender—the entire email was inviting, and gave the recipient the opportunity to begin a conversation with a person, rather than receive marketing from an organization. Compassion saw that the treatment email, in addition to the lift in open rate, received 7x more submissions to the program than the control, and facilitated a tremendous amount of inbound responses beyond that, proving that personalization helps transcend “marketing”.