Compassion International

How a personalizing the “envelope” of an email affects open rate

Experiment ID: #21354

Compassion International

Experiment Summary

Ended On: 10/24/2019

Compassion International invited young college graduates to join their Compassion Go internship program through an email offer. The email came from “Compassion International” as the sender, and they wanted to know if personalizing the “envelope” on the email could increase open rate. They changed the sender to the name of an intern, and changed the subject line from a very sales-heavy subject line to “Quick question”. Then, they ran a test to see which one produced a higher open rate. 

Research Question

How will personalizing the “envelope” of an email affect open rate?

Results

Treatment Name Open Rate Relative Difference Confidence
C: 12.0%
T1: Personal Style 51.4% 329.8% 100.0%

This experiment has a required sample size of 11 in order to be valid. Since the experiment had a total sample size of 50,668, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    329.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment email had a massive 329% increase in open rate. However, this “personal style” wasn’t just applied to the subject line and sender—the entire email was inviting, and gave the recipient the opportunity to begin a conversation with a person, rather than receive marketing from an organization. Compassion saw that the treatment email, in addition to the lift in open rate, received 7x more submissions to the program than the control, and facilitated a tremendous amount of inbound responses beyond that, proving that personalization helps transcend “marketing”. 


Experiment Documented by Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.

Question about experiment #6385

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.