How the style of offer impacts email acquisition Experiment ID: #8595

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 2/13/2018 - 2/16/2018

Leading up to the oral arguments for the National Institute of Family and Life Advocates vs. Becerra, Alliance Defending Freedom wanted to discover the ideal offer and message to use with this case. The week of the oral arguments will provide a sizable audience for the organization, so the goal is to identify the best tactic to make the most of the situation.

Previous experiments had identified that a Freedom of Speech angle was the ideal messaging approach for the case. However, we felt that there might be a better offer than the existing “News Updates on the Case.” We decided to test out a Statement of Belief that would allow people to come together and express their support in favor of the defendants.

Research Question

Will a “Statement of Belief” be a more appealing offer than the existing “News Updates?”

Design

C: Latest News
T1: Statement of Belief

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Latest News 16.5%
T1: Statement of Belief 29.8% 80.7% 100.0%

This experiment has a required sample size of 78 in order to be valid. Since the experiment had a total sample size of 1,029, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 80.7% increase in conversion rate
× 0% increase in average gift

Key Learnings

After running for 3 days, we saw a statistically significant 80% lift in conversion to the Statement of Belief offer. This is a significant win because it will allow us to acquire emails for roughly half the cost which will be even more impactful once this case is in the national spotlight. This win will also allow us to shift our focus back to the FB ads; we can now run targeted experiments that drive more motivated visitors since the offer is now the same for every visitor.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.





Related Experiments

This experiment is part of a series of experiments aimed at improving overall results. Take a look at some of the other iterations:


Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.