How both a campaign slidedown and slideout impacts email acquisition rate on blog pages Experiment ID: #7906

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 10/30/2017 - 11/21/2017

Alliance Defending Freedom recently found that adding a campaign donation slidedown to their homepage increased donor conversion by nearly 500%. As they continued to introduce this approach across their site, they decided to isolate traffic to their blog pages and conduct a similar test. This time they wondered: Will campaign slidedowns and slideouts that encourage the site visitor to sign up for updates on the Jack Phillips case increase email acquisition rate? They created a treatment version that included these campaign elements and launched an A/B test to find out.

Research Question

Will a campaign-specific slidedown and slideout increase email acquisition rate on blog pages?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: No Interrupters
T1: Email Interrupters
T2: Donation Interruptors

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: No Interrupters 0.03%
T1: Email Interrupters 0.70% 2,001.4% 100.0%
T2: Donation Interruptors 0.04% 11.9% 14.7%

This experiment has a required sample size of 655 in order to be valid. Since the experiment had a total sample size of 46,408, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 2,001.4% increase in conversion rate
× 0% increase in average gift

Key Learnings

Alliance Defending Freedom found that the treatment version increased email acquisition by 2,000%. It’s clear these campaign-specific slidedowns and slideouts were effective at converting website traffic into motivated fans and potential donors of ADF. While this test is supported by many of the learnings produced in the prior homepage slidedown test we discussed earlier, there are some key differences and new findings worth mentioning. First, it’s important to differentiate between the donation slidedown and the email slidedown and slideout. While the donation slidedown used a CTA button to drive traffic to a ‘Jack Phillips’ donation page, the email acquisition slidedown and slideout contained a email form with no clickthrough opportunity. By removing a step in the visitor’s pathway (i.e. keeping the visitor on the same page), we found that this version reduced friction and allowed motivated visitors to take action immediately by signing up for updates. We can use this tactic here, since the level of commitment for a donation is higher than an email address and requires more information.

The addition of the campaign slideout also deserves credit for this significant increase. On the homepage slidedown test, visitors were only given the option of clicking through to the donation page at the very top of the site. If they scrolled down quickly, they would not see this option again. In this email acquisition version, the slidedown still served as the primary call-to-action. However, if a visitor scrolled down past 50% of the page, they would also see the slideout. This addition reintroduced the offer seen at the top of the page, adding urgency to the campaign offer.

Finally, this test takes a good step in determining the difference between the nature of homepage visitors and blog page visitors. It appears here that blog page visitors were highly motivated, when presented an email acquisition offer. However, this test also re-validated that the Alliance Defending Freedom blog is not a great direct source for donations. Further testing is being conducted to hone in on the particularities between blog visitors and homepage visitors.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Grant Klembara

Grant is an Optimization Director. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.