Alliance Defending Freedom

How an ‘Up Next’ Slideout Impacts Mobile Clickthrough Rate

Experiment ID: #6887

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 06/16/2017 - 06/30/2017

Alliance Defending Freedom noticed that their most engaged users–and consequently, the ones that were most likely to become supporters–primarily spent their time reading Alliance Defending Freedom’s blogs. Noticing this as a major step in the donor journey, they decided to test a ‘Up Next’ slideout designed to keep the site visitor engaged with a new article immediately after finishing one. If they could keep their visitor engaged just a little bit longer, they knew they could increase that user’s affinity for their mission–and perhaps, their willingness to support Alliance Defending Freedom. They set up an A/B test to test this hypothesis.

Research Question

Will providing a single article that is “Up Next” increase clickthrough rate on mobile devices?

Design

C: Without Slideout
T1: With Slideout

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Without Slideout 3.6%
T1: With Slideout 4.7%29.6% 100.0%

This experiment has a required sample size of 2,667 in order to be valid. Since the experiment had a total sample size of 26,578, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    29.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

Users that were motivated enough to finish reading the entire blog article responded favorably to the ‘Up Next’ slideout. Over the course of two weeks, Alliance Defending Freedom noticed that the treatment did, in fact, increase engagement by nearly 30%. It’s evident that readers wanted more to get more content. By removing the friction of searching for the next blog to read or deciding which topic to choose, Alliance Defending Freedom found that visitors stayed engaged for a longer period of time, especially when the next article was chosen for them. This finding has prompted further side-by-side funnel testing to determine what kind of funnel is the most effective for user engagement. There is also room for similar tests measuring results all the way through the funnel, to view the effects of email acquisition and donor conversion rates.


Experiment Documented by NextAfter

Question about experiment #6887

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.