How a value proposition built upon “what” instead of “why” impacts email acquisition
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 02/09/2016 - 02/15/2016
Dallas Theological Seminary had recently launched their first online course, a study of the Gospel of John. The course was offered for free in exchange for a visitor’s name, email, and address. With significant traffic being sent to the site, we wanted to find the best way to communicate the value to the visitors.
Using the learnings from experiments run with similar organization, we tried applying two key principles to the copy on the page:
- We emphasized what the visitor would get by subscribing instead of what they would have to do
- We reduced the friction associated with the process by removing copy that we felt was unnecessary to the conversion.
We then tested these two pages against each other and monitored results.
Research Question
Can we increase email acquisition by limiting friction and emphasizing the value to the visitor?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Original | 37.5% | ||
| T1: | Less Friction / Value to Visitor | 35.2% | -6.1% | 96.2% |
This experiment has a required sample size of 3,396 in order to be valid. Since the experiment had a total sample size of 7,583, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 6.1% decrease in conversion rate
× 0% increase in average gift
Key Learnings
The treatment ended up decreasing email acquisition by 6%. Our hypothesis is that this was not caused by reducing friction or emphasizing the value to the visitor but because we removed the “why” behind the course and focused solely on the “what.”
The paragraph removed talked about the mission of DTS and it allowed visitors to the page to align around this same mission and mentally put themselves in the same audience. Those that felt they belonged would be more likely to take the desired action of signing up.