How the emphasis of the ad influences email acquisition Experiment ID: #8399
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Timeframe: 1/19/2018 - 1/22/2018
Alliance Defending Freedom was launching an offer around one of their upcoming cases that was heading to the Supreme Court. The case was interesting in that it touched upon the two “hot topic” issues of Freedom of Speech and Pro-Life advocacy. We wanted to determine which messaging angle was the most effective with regards to email acquisition.
Which messaging approach, “pro-life” or “freedom of speech” will be the most effective at acquiring email addresses?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|T1:||Freedom of Speech||0.44%||79.8%||100.0%|
This experiment has a required sample size of 7,642 in order to be valid. Since the experiment had a total sample size of 56,173, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 79.8% increase in conversion rate
× 0% increase in average gift
We found that the freedom of speech message received 80% more emails than the pro-life message. Additionally, people were 94% more likely to click on the freedom of speech ad and we were able to acquire emails for 78% of the cost when compared to the pro-life message.
We have plans to further validated these results with an email acquisition experiment on the landing page. Once we know definitively the messaging approach to use with this case, we will be able to increase out overall emphasis on acquisition.