How the disruptive placement of an offer affects email acquisition Experiment ID: #2542

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 11/4/2015 - 11/10/2015

In the continued efforts to increase the engagement on the blog posts for Alliance Defending Freedom, we wanted to find ways to further improve the number of emails acquired or donations received. Our previous experiments had shown that a softer call-to-action was the best language to use so we used this as our control.

We had a hypothesis that by using two equally weighted calls-to-action at the bottom of the post, we may be adding unnecessary friction to the process. There is friction associated with making it to the bottom of the page and with making the visitors choose between the two actions.

We decided to create an experiment that would reduce the calls-to-action so we were only focusing on one, the email acquisition. We also wanted to move it from the bottom of the page and make it more noticeable to the visitor. As a result, we made the offer slide in from the side of the page once the reader had made it to the bottom of the blog post.

Research Question

Which method for delivering the email acquisition offer will produce the best conversion rate?

Design

C: Inline Calls-to-Action
T1: Slideout Email Capture

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Inline Calls-to-Action 0.07%
T1: Slideout Email Capture 0.42% 523.9% 100.0%

This experiment has a required sample size of 1,476 in order to be valid. Since the experiment had a total sample size of 11,623, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 523.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

By reducing the number of calls-to-action to just one and making the offer disruptive to the reading process, we were able to increase the email acquisition rate by 523.9%. It is our hypothesis that by leaving the offers at the bottom of the page, they were getting lost in the other options found down there. By creating one clear call-to-action, we were able to add clarity to the process and guide visitors toward the action we most wanted them to take.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.





Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.


Pick your format of choice:

Related Experiments

This experiment is part of a series of experiments aimed at improving overall results. Take a look at some of the other iterations:


Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.