How extended value proposition email copy affects donor conversion Experiment ID: #7821

Leadership Institute

Experiment Summary

Timeframe: 11/4/2017 - 11/6/2017

The Leadership Institute recently sent a letter to their subscribers about the ‘Normandy Coalition’ –a new group of conservative and liberty-minded student organizations, policy groups, and legal defense foundations. As they developed this email copy, they wondered: Will a substantially longer email that includes more stories, examples, and value proposition language, increase motivation and donor conversion? They created a treatment version of the email and launched an A/B test to find out.

Research Question

Will extended value proposition language increase donor conversion in an appeal email?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: Control
T1: Treatment 1

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 18.2%
T1: Treatment 1 37.5% 106.3% 99.4%

This experiment has a required sample size of 40 in order to be valid. Since the experiment had a total sample size of 160, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 106.3% increase in conversion rate
× 0% increase in average gift

Key Learnings

The Leadership Institute found that the longer treatment version did, in fact, increase donor conversion rate by 106%. It’s clear that adding more language supporting the value proposition of this campaign increased motivation and clarity. This is a substantial increase in email copy–from 550 words to 2,200 words–and provides a valuable learning for us when thinking through how long an appeal email should be. This test clearly shows that offering the email recipient a larger selection of evidence (i.e. stories and examples) only helped build the momentum the reader was experiencing as they clicked through to the donation page.

This is a test that the Leadership Institute is using and applying to upcoming campaigns, and it is one that will continue to help them develop a better understanding of who there donors are. It is also a test that is prompting further experiments by isolating the variety of variables in this test to see which one has the biggest impact (including story type, regional vs. national examples, and more).


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Grant Klembara

Grant is an Optimization Director. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.