How value proposition affects conversion rate
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 01/25/2016 - 02/09/2016
Dallas Theological Seminary had a great response to their first free online course on the Gospel of John. But objectively looking at the headline 0f their signup page a few weeks later, they wondered if the value proposition could be improved — specifically, by using something other than the word “study”.
Other experiments had shown that offering the benefits of the offer often outperform words like “study” that make the offer seen like work. So they launched two headline tests to see if they could affect conversion. Both headlines sought to show the real opportunity of the course — to “step inside the classroom”, but with different desired effects. The first headline offered the opportunity to “see the gospel of John like never before” and the other to “grow your understanding of John’s gospel”.
They launched an A/B/C test to determine a winner.
Research Question
Will reducing perception of cost and work increase conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Original | 32.8% | ||
| T1: | Variation #2 | 32.4% | -1.1% | 39.5% |
| T2: | Variation #1 | 30.6% | -6.5% | 86.6% |
This experiment has a required sample size of 5,606 in order to be valid. Since the experiment had a total sample size of 19,056, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
Surprisingly, neither variation outperformed the control. This tells us something very interesting — “study” might actually be a positive factor of the value proposition. It suggests that prospects for this course might be looking to “study” and really spend time with the course.
It may also suggest that the value proposition to “step inside the classroom” is unclear or undesirable. This concept lends itself to further testing.