Alliance Defending Freedom

How 3rd party validation can impact visitor engagement

Experiment ID: #6032

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 01/06/2017 - 01/27/2017

On the mobile version of ADF’s blog, we have focused significant efforts into finding ways to increase visitor engagement. The majority of visitors are coming from social media and often do not usually explore the site beyond the one blog post that they read.

Our previous tests had been unsuccessful in increasing the percentage of visitors that continue to explore the site. But, by analyzing their results, we had developed a new hypothesis that we could increase engagement by increasing the perceived popularity of the posts. We believe that by ordering the recent posts with the most popular at the top would increase the likelihood of a visitor reading the article.

Research Question

Will incorporating 3rd party validation increase the likelihood of a visitor to read another article?

Design

C: Most Recent Articles
T1: Most Popular Articles

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Most Recent Articles 43.8%
T1: Most Popular Articles 44.9%2.5% 100.0%

This experiment has a required sample size of 15,902 in order to be valid. Since the experiment had a total sample size of 143,521, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    2.5% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

While not a huge lift, we did see a statistically significant 2.5% increase to the number of visitors that read another article. This would imply that visitors were more likely to be influenced to read a post if they knew that other visitors had done the same thing. This is a good example of subtly using 3rd party credibility to increase engagement.


Experiment Documented by NextAfter

Question about experiment #6032

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.