How the style of offer impacts email acquisition
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 02/13/2018 - 02/16/2018
Leading up to the oral arguments for the National Institute of Family and Life Advocates vs. Becerra, Alliance Defending Freedom wanted to discover the ideal offer and message to use with this case. The week of the oral arguments will provide a sizable audience for the organization, so the goal is to identify the best tactic to make the most of the situation.
Previous experiments had identified that a Freedom of Speech angle was the ideal messaging approach for the case. However, we felt that there might be a better offer than the existing “News Updates on the Case.” We decided to test out a Statement of Belief that would allow people to come together and express their support in favor of the defendants.
Research Question
Will a “Statement of Belief” be a more appealing offer than the existing “News Updates?”
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Latest News | 16.5% | ||
| T1: | Statement of Belief | 29.8% | 80.7% | 100.0% |
This experiment has a required sample size of 78 in order to be valid. Since the experiment had a total sample size of 1,029, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 80.7% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running for 3 days, we saw a statistically significant 80% lift in conversion to the Statement of Belief offer. This is a significant win because it will allow us to acquire emails for roughly half the cost which will be even more impactful once this case is in the national spotlight. This win will also allow us to shift our focus back to the FB ads; we can now run targeted experiments that drive more motivated visitors since the offer is now the same for every visitor.