How a banner header image impacts article engagement for desktop users
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 10/26/2017 - 11/30/2017
Alliance Defending Freedom was interested in knowing how blog page design impacted reader engagement on their site. They wondered: Will removing the header image on blog pages decrease friction and increase reader engagement? They created a trackable goal that measured clicks from one blog to a second article. They then created a treatment version that removed the header image and launched an A/B test to find out.
Research Question
Will removing the header image on blog pages increase reader engagement?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 10.7% | ||
| T1: | Treatment 1 | 9.8% | -8.3% | 69.4% |
This experiment has a required sample size of 8,954 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Alliance Defending Freedom found that the removal of the blog header image did not significantly impact reader engagement on their blog. The treatment page, in other words, did not produce a statistically significant increase or decrease in reader engagement for desktop users.
This test does, however, seek to address an important question–one that further testing will continue to try and answer. That question is: How does design impact page functionality and usability? This is an important, ongoing question that every nonprofit marketer should be motivated to answer for their organization because both form and function must exist in a healthy balance.