How the kind of images used in a Facebook ad impacts conversion
Harvest Ministries
Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.
Experiment Summary
Timeframe: 02/07/2017 - 02/13/2017
Harvest Ministries was running a Facebook campaign for a daily devotional focused on the topic of family. They knew that images of Greg Laurie, the lead pastor and founder of Harvest, perform better in the ads than images of people less recognizable. For this campaign, the control was an image they had used before and had success with. It was a black and white image of Greg and his wife, Cathe. For the experiment they wondered if a colored version of that same image would perform better or if an image of the couple looking straight at the camera would have greater success.
Research Question
Which image would help acquire more email addresses?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Control | 0.38% | ||
T1: | Treatment 1 | 0.25% | -35.1% | 100.0% |
T2: | Treatment 2 | 0.31% | -19.7% | 99.8% |
This experiment has a required sample size of 21,236 in order to be valid. Since the experiment had a total sample size of 338,155, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 35.1% decrease in conversion rate
× 0% increase in average gift
Key Learnings
To our surprise, the black and white image of Greg and Cathe performed best of the three images. In tests from other organizations, we have found that images with color perform better than black and white. What we can take away from this test is:
- The black and white image may have stood out and was more aesthetically pleasing to people.
- The image of Greg and Cathe looking at the camera straight on didn’t convey the warmth that the image of them holding hands did.
Question about experiment #6158
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.