How extended value proposition email copy affects donor conversion
Leadership Institute
Experiment Summary
Timeframe: 11/04/2017 - 11/06/2017
The Leadership Institute recently sent a letter to their subscribers about the ‘Normandy Coalition’ –a new group of conservative and liberty-minded student organizations, policy groups, and legal defense foundations. As they developed this email copy, they wondered: Will a substantially longer email that includes more stories, examples, and value proposition language, increase motivation and donor conversion? They created a treatment version of the email and launched an A/B test to find out.
Research Question
Will extended value proposition language increase donor conversion in an appeal email?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 18.2% | ||
| T1: | Treatment 1 | 37.5% | 106.3% | 99.4% |
This experiment has a required sample size of 40 in order to be valid. Since the experiment had a total sample size of 160, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 106.3% increase in conversion rate
× 0% increase in average gift
Key Learnings
The Leadership Institute found that the longer treatment version did, in fact, increase donor conversion rate by 106%. It’s clear that adding more language supporting the value proposition of this campaign increased motivation and clarity. This is a substantial increase in email copy–from 550 words to 2,200 words–and provides a valuable learning for us when thinking through how long an appeal email should be. This test clearly shows that offering the email recipient a larger selection of evidence (i.e. stories and examples) only helped build the momentum the reader was experiencing as they clicked through to the donation page.
This is a test that the Leadership Institute is using and applying to upcoming campaigns, and it is one that will continue to help them develop a better understanding of who their donors are. It is also a test that is prompting further experiments by isolating the variety of variables in this test to see which one has the biggest impact (including story type, regional vs. national examples, and more).