How an ‘Up Next’ Slideout Impacts Mobile Clickthrough Rate
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 06/16/2017 - 06/30/2017
Alliance Defending Freedom noticed that their most engaged users–and consequently, the ones that were most likely to become supporters–primarily spent their time reading Alliance Defending Freedom’s blogs. Noticing this as a major step in the donor journey, they decided to test a ‘Up Next’ slideout designed to keep the site visitor engaged with a new article immediately after finishing one. If they could keep their visitor engaged just a little bit longer, they knew they could increase that user’s affinity for their mission–and perhaps, their willingness to support Alliance Defending Freedom. They set up an A/B test to test this hypothesis.
Research Question
Will providing a single article that is “Up Next” increase clickthrough rate on mobile devices?
Design
Results
Treatment Name | Click Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Without Slideout | 3.6% | ||
T1: | With Slideout | 4.7% | 29.6% | 100.0% |
This experiment has a required sample size of 2,667 in order to be valid. Since the experiment had a total sample size of 26,578, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
29.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
Users that were motivated enough to finish reading the entire blog article responded favorably to the ‘Up Next’ slideout. Over the course of two weeks, Alliance Defending Freedom noticed that the treatment did, in fact, increase engagement by nearly 30%. It’s evident that readers wanted more to get more content. By removing the friction of searching for the next blog to read or deciding which topic to choose, Alliance Defending Freedom found that visitors stayed engaged for a longer period of time, especially when the next article was chosen for them. This finding has prompted further side-by-side funnel testing to determine what kind of funnel is the most effective for user engagement. There is also room for similar tests measuring results all the way through the funnel, to view the effects of email acquisition and donor conversion rates.
Question about experiment #6887
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.