How the disruptive placement of an offer affects email acquisition Experiment ID: #2542
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Timeframe: 11/4/2015 - 11/10/2015
In the continued efforts to increase the engagement on the blog posts for Alliance Defending Freedom, we wanted to find ways to further improve the number of emails acquired or donations received. Our previous experiments had shown that a softer call-to-action was the best language to use so we used this as our control.
We had a hypothesis that by using two equally weighted calls-to-action at the bottom of the post, we may be adding unnecessary friction to the process. There is friction associated with making it to the bottom of the page and with making the visitors choose between the two actions.
We decided to create an experiment that would reduce the calls-to-action so we were only focusing on one, the email acquisition. We also wanted to move it from the bottom of the page and make it more noticeable to the visitor. As a result, we made the offer slide in from the side of the page once the reader had made it to the bottom of the blog post.
Which method for delivering the email acquisition offer will produce the best conversion rate?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|T1:||Slideout Email Capture||0.42%||523.9%||100.0%|
This experiment has a required sample size of 1,476 in order to be valid. Since the experiment had a total sample size of 11,623, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 523.9% increase in conversion rate
× 0% increase in average gift
By reducing the number of calls-to-action to just one and making the offer disruptive to the reading process, we were able to increase the email acquisition rate by 523.9%. It is our hypothesis that by leaving the offers at the bottom of the page, they were getting lost in the other options found down there. By creating one clear call-to-action, we were able to add clarity to the process and guide visitors toward the action we most wanted them to take.