Harvest Ministries

How the kind of images used in a Facebook ad impacts conversion

Experiment ID: #6158

Harvest Ministries

Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.

Experiment Summary

Timeframe: 02/07/2017 - 02/13/2017

Harvest Ministries was running a Facebook campaign for a daily devotional focused on the topic of family. They knew that images of Greg Laurie, the lead pastor and founder of Harvest, perform better in the ads than images of people less recognizable. For this campaign, the control was an image they had used before and had success with. It was a black and white image of Greg and his wife, Cathe. For the experiment they wondered if a colored version of that same image would perform better or if an image of the couple looking straight at the camera would have greater success.

Research Question

Which image would help acquire more email addresses?

Design

C: Control
T1: Treatment 1
T2: Treatment 2

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.38%
T1: Treatment 1 0.25%-35.1% 100.0%
T2: Treatment 2 0.31%-19.7% 99.8%

This experiment has a required sample size of 21,236 in order to be valid. Since the experiment had a total sample size of 338,155, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 35.1% decrease in conversion rate
× 0% increase in average gift

Key Learnings

To our surprise, the black and white image of Greg and Cathe performed best of the three images. In tests from other organizations, we have found that images with color perform better than black and white. What we can take away from this test is:

  1. The black and white image may have stood out and was more aesthetically pleasing to people.
  2. The image of Greg and Cathe looking at the camera straight on didn’t convey the warmth that the image of them holding hands did.


Experiment Documented by Courtney Gaines
Courtney Gaines is Vice President at NextAfter.

Question about experiment #6158

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.