How strengthening clarity for the call to action increased donor conversion from an email Experiment ID: #19000

Billy Graham Evangelistic Association

Experiment Summary

Ended On: 10/29/2019

Billy Graham Evangelical Association send out a monthly email called, “From the Desk of Franklin Graham”. This email serves two purposes – one, to ask people to pray, and two, to ask people to make a gift. When we analyzed what was currently being sent out, we hypothesized that a more simplified approach to the email may increase conversion. We hypothesized that making the email look and feel like it was actually from Franklin Graham may increase engagement and ultimately conversion. To do this, we removed the template of the email and made it more plain text, removed the headline, image, CTA’s buttons, and social sharing options. We then moved the premium off up into the main part of the body copy to connect the donation ask to the offer. No kept the same body copy of the email and didn’t add or change any of it. This approach was tested on 10% of the email file.

Research Question

Can we increase donor conversion with a radical, simplified version of the email?

Design

C: From the Desk of Franklin Graham Control
T1: From the Desk of Franklin Graham Treatment

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: From the Desk of Franklin Graham Control 0.08%
T1: From the Desk of Franklin Graham Treatment 0.14% 74.9% 100.0%

This experiment has a required sample size of 22,773 in order to be valid. Since the experiment had a total sample size of 291,317, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 74.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment version increased donor conversion by 75%! By simplifying the email and not making it feel like a templated, marketing email, people were more highly motivated to give. It removed all of the elements that distracted the reader from the message itself and clearly communicated what it was that we wanted them to do.


Experiment Documented by...

Courtney Gaines

Courtney is the Senior Director of Optimization at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.