How clarifying the call-to-action for mobile visitors affects engagement
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 04/04/2016 - 04/20/2016
Alliance Defending Freedom has a unique homepage banner that allows for multiple stories, cases, and appeals to be featured. The banner shows the featured image and includes the headline on top of it. When a user hovers over the image, more details of the article appears including the category and a call to action such as “Learn More” or “Double Your Impact.”
While discussing ways to increase the engagement of the visitors to the site, it was realized that mobile users would never see the call-to-action. It is only visible on hover which is a function not available on phones and tablets. We decided to run a test targeting mobile devices to see if moving the call-to-action below the headline on the image would encourage more users to explore the website.
Research Question
Does including the call-to-action on the homepage banner for mobile devices increase the number of people that explore the website?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Call-To-Action on Hover | 31.2% | ||
| T1: | Call-To-Action over Banner | 34.9% | 11.5% | 92.8% |
This experiment has a required sample size of 1,306 in order to be valid. Since the experiment had a total sample size of 2,204, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
By moving the call-to-action on top of the banner; we were able to increase the number of people exploring the website by 11.5%. Our hypothesis is that without the call-to-action, the visitors were unclear on what they should do next. It made it more obvious that the banners should be clicked on and it clarified what their next steps would be on the subsequent page.