How a campaign slidedown impacts donor conversion on the homepage
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 10/17/2017 - 10/30/2017
Alliance Defending Freedom is preparing to defend Jack Phillips in front of the Supreme Court later this year. They know this case will require a lot of additional resources and support, since it has reached a national stage and could have a lasting impact for religious liberty. Interested in finding a new way to alert their visitors to the magnitude of Jack’s case, they wondered: Will a homepage slidedown that directs traffic to a Jack-specific donation page increase the motivation of site visitors to make a gift?
They launched an A/B test on the homepage to find out.
Research Question
Will a specific campaign slidedown increase donor conversion on the homepage?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Slidedown | 0.21% | ||
| T1: | Donation Slidedown | 1.2% | 474.4% | 99.9% |
This experiment has a required sample size of 564 in order to be valid. Since the experiment had a total sample size of 2,979, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 474.4% increase in conversion rate
× 0% increase in average gift
Key Learnings
Alliance Defending Freedom found that the homepage slidedown offer increased donor conversion by a dramatic 474%. This test clearly indicates that the homepage slidedown aligned with visitor motivation, and there are a couple of different explanations for us to understand and review. First, it’s apparent that visitor’s were positively impacted by three main factors: (1) the slidedown’s visible movement (2) a direct callout of the magnitude of this case and (2) a ‘Double your gift’ match announcement. The movement of this slideout immediately caught the user’s attention. Once they were engaged, the second factor signaled the national importance of this case. Once the visitor understood this case’s potential impact, the final factor signaled the existing commitment from other Alliance Defending Freedom donors to this case. Reduced friction for visitors –a simple call-to-action and a clickthrough opportunity that required no searching–allowed motivated visitors to quickly find the donation page.
These results are being used to produce other similar tests across the organization. It’s interesting to note that this slidedown on the homepage produced far more dramatic results than on other parts of the site (such as ‘Blog’ and the ‘About Us’ pages). This reinforces the assumption that homepage visitors are far more motivated than any other segment of visitors on a site.