How 3rd party validation can impact visitor engagement
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 01/06/2017 - 01/27/2017
On the mobile version of ADF’s blog, we have focused significant efforts into finding ways to increase visitor engagement. The majority of visitors are coming from social media and often do not usually explore the site beyond the one blog post that they read.
Our previous tests had been unsuccessful in increasing the percentage of visitors that continue to explore the site. But, by analyzing their results, we had developed a new hypothesis that we could increase engagement by increasing the perceived popularity of the posts. We believe that by ordering the recent posts with the most popular at the top would increase the likelihood of a visitor reading the article.
Research Question
Will incorporating 3rd party validation increase the likelihood of a visitor to read another article?
Design
Results
Treatment Name | Click Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Most Recent Articles | 43.8% | ||
T1: | Most Popular Articles | 44.9% | 2.5% | 100.0% |
This experiment has a required sample size of 15,902 in order to be valid. Since the experiment had a total sample size of 143,521, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
2.5% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
While not a huge lift, we did see a statistically significant 2.5% increase to the number of visitors that read another article. This would imply that visitors were more likely to be influenced to read a post if they knew that other visitors had done the same thing. This is a good example of subtly using 3rd party credibility to increase engagement.
Question about experiment #6032
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.