How increasing the force of the value proposition affects email acquisition
Leadership Institute
Experiment Summary
Timeframe: 11/14/2017 - 02/06/2018
CampusReform, an initiative of the Leadership Institute, garnered significant traffic to their content based on the timeliness and relevancy of their content. However, the majority of the traffic did not convert into email addresses for ongoing cultivation.
Their team looked at the email subscription form and noticed several elements that could be optimized. The headline said exactly what the offer was: “Newsletter”, rather than a compelling value proposition. The supporting copy told them nothing about what they were going to get. There were social share icons in between the supporting copy and the form itself. And finally, the button copy was placed on a red background—which connotes fear or “stop”.
They created a treatment with a new headline which reflected a possible motivation of the reader (“Stay Informed”), copy that told them what they would receive when signing up, removed the social share icons, and turned the button from red to green, a color commonly connoted with “acceptance” and “go”.
They launched an A/B test to determine a winner.
Research Question
Will increasing the force of the value proposition increase email acquisition rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.04% | ||
| T1: | Treatment 1 | 0.06% | 27.9% | 99.9% |
This experiment has a required sample size of 261,203 in order to be valid. Since the experiment had a total sample size of 1,513,445, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 27.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment, with the four significant changes designed to intensify the force of the value proposition and reduce friction, increased email acquisition by 28%.
Because of the traffic to CampusReform.com, this has a significant opportunity to increase the number of emails acquired. However, the placement of the email acquisition offer might be affecting how many people even see the offer. Future testing should place this email acquisition form inline into the copy, especially for mobile users who don’t see the sidebar.