How including an image in a personal email affects donor conversion Experiment ID: #6272

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 3/1/2017 - 3/10/2017

Alliance Defending Freedom was raising money for a very specific case. They were renting email lists for the first time, and so they knew that, to many of the recipients, they were introducing the person they were defending for the first time. However, since this case made national news, they knew that many of the recipients would be familiar with the case, if not the name of the defendant.

They had tested several different treatments and had arrived at a very plain text email with no formatting. However, since they were introducing someone new, they thought that it might produce a lift if they included a picture of Barronelle, the defendant. Since this was truly an introduction, then they thought the reader might be more likely to respond if they could see her before they reached the landing page, and when they reached the landing page, they might be more likely to convert.

They created a treatment with an image and split the next email list rental in half to determine a winner.

Research Question

Will including an image in a personal email increase donor conversion?


C: No Image
T1: With image


Treatment Name Conv. Rate Relative Difference Confidence
C: No Image 0.05%
T1: With image 0.09% 80.8% 98.6%

This experiment has a required sample size of 31,771 in order to be valid. Since the experiment had a total sample size of 100,000, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 80.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment with the image had a significant effect—not only a big lift in clickthrough rate, but also a big lift in donations—80.8%.

This illustrates the power of imagery as essential to understanding the value proposition expressed in the email. While a previous email template had proven to reduce conversion, it was because the imagery was not relevant to the value proposition and instead added unnecessary friction. However the image of Barronelle in this evoked a powerful emotion—in this case, sympathy—which increased the force of the value proposition and even triggered a higher motivation to continue through with the donation.

This treatment was quickly adopted as the new control, and further testing began to try to personalize it even more without losing the directness of the message.

Related Experiments

This experiment is part of a series of experiments aimed at improving overall results. Take a look at some of the other iterations:

Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.