How simplifying the call-to-action on a page impacts email acquisition
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 11/30/2017 - 12/05/2017
Research Question
Will a simplified landing page help improve email acquisition rates?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Original | 2.1% | ||
| T1: | Simplified Version | 3.1% | 48.3% | 100.0% |
This experiment has a required sample size of 1,886 in order to be valid. Since the experiment had a total sample size of 13,850, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 48.3% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment saw a 48% increase to the number of emails acquired. This emphasizes the need to focus a landing page on a single call-to-action when there is one primary metric to influence. In this example, the control had four different actions it was requesting the user to take which took the emphasis of the primary action of subscriber for email.
As a note, the original treatment included an exit intent that the treatment did not have (the result of an oversight in implementation). This means that the impact of the simpler version would likely have been even greater had this been included on that page.