How a simplified and user-focused value proposition affects sign-up rate
The Navigators
Experiment Summary
Timeframe: 07/25/2023 - 08/16/2023
After a previous test on the DDJ landing page that tested into layout, The Navigators wanted to turn their attention to testing the elements themselves. In this copy test, we hypothesized framing the language around the user and simplifying the copy to their desired impact rather than the Navigators would increase sign-up rate. Rather than answering why the organization created Digital Discipleship, the treatment is focused on asking questions directly to the user and positioning the resource as the answer.
Research Question
We believe that a simplified value prop about the user’s journey and desired impact for the DDJ landing page will achieve an increase in sign-up rate because this will better communicate the value and focus on answering why they should be interested in the offer.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 37.2% | ||
| T1: | Simplified Value Prop | 34.7% | -6.9% | 29.3% |
This experiment has a required sample size of 2,655 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The key learning from this experiment is that simplifyign the value prop did not achieve the desired increase in sign-up rate for the DDJ landing page. The treatment actually saw a decrease in Email Acquisition for All Traffic compared to the control, however this was not at a valid level of confidence. More testing is required.
Some questions to consider are the ways in which this framing shifted the responsibility onto the reader. Although the attempt was to not focus on why the organization created the resource so much as the user’s journey and the impact they want to make — it’s possible this put too much pressure on the reader. What if they aren’t ready to disciple or commit to “come alongside someone?” The original used these terms, but only in the sense of whether the readers themselves has ever had someone do this for them, not whether they want to do this for someone else.
Future experimentation should continue to test different approaches to framing the language and offer on the DDJ landing page to determine the most effective strategy for increasing sign-up rates. It is important to gather more data and achieve a higher statistical level of confidence in order to make valid conclusions.