Alliance Defending Freedom

How the emphasis of the ad influences email acquisition

Experiment ID: #3047

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 01/19/2018 - 01/22/2018

Alliance Defending Freedom was launching an offer around one of their upcoming cases that was heading to the Supreme Court. The case was interesting in that it touched upon the two “hot topic” issues of Freedom of Speech and Pro-Life advocacy. We wanted to determine which messaging angle was the most effective with regards to email acquisition.

Research Question

Which messaging approach, “pro-life” or “freedom of speech” will be the most effective at acquiring email addresses?

Design

C: Pro-Life
T1: Freedom of Speech

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Pro-Life 0.25%
T1: Freedom of Speech 0.44%79.8% 100.0%

This experiment has a required sample size of 7,642 in order to be valid. Since the experiment had a total sample size of 56,173, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 79.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

We found that the freedom of speech message received 80% more emails than the pro-life message. Additionally, people were 94% more likely to click on the freedom of speech ad and we were able to acquire emails for 78% of the cost when compared to the pro-life message.

We have plans to further validated these results with an email acquisition experiment on the landing page. Once we know definitively the messaging approach to use with this case, we will be able to increase out overall emphasis on acquisition.


Experiment Documented by NextAfter

Question about experiment #3047

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.