How reducing the perceived investment of a call to action impacts account creation rate (Pt.3)
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Timeframe: 06/14/2023 - 07/02/2023
CaringBridge has been running ebook offers where they prompt the ebook downloader to create an account with their organization. They have previously used the call to action, “Create a site” to communicate the breadth of what CaringBridge offers when you create an account to share yours our your loved ones health journey. Our hypothesis was that the language to “create a site” may have the perception that they’re about to make a large commitment. Creating a site may sound tedious, laborious, or intimidating. We wanted to reduce any anxiety about the process by simply changing the word “site” to “page.” This phrase is still accurate but reduces the perceived investment and commitment they’re about to make.
When we ran the first iteration of this test we ran the experiment on one offer. We saw a valid lift when using “page.” Before rolling it out to all offers, we decided to test it among all offers and add another variant using “journal” to see if that would also reduce anxiety. In the second experiment, we saw a decrease in conversion rate when using “page.”
In this particular experiment we tested this concept on a separate offer that used slightly different verbiage in the button and started with the language “page” under the first assumption that people responded more to the use of “page”. Therefore, the control is “page,” treatment 1 is “site,” and treatment 2 is “journal.”
Research Question
We believe that prompting people to create a “page” or “journal” instead of a “site” will increase site creation rate because there will be a reduced perception of work involved which will reduce anxiety as well.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 2.6% | ||
| T1: | Start a Site | 5.1% | 93.6% | 80.7% |
| T2: | Start a Journal | 6.3% | 138.8% | 91.7% |
This experiment has a required sample size of 389 in order to be valid. Since the experiment had a total sample size of 600, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
The key learning from this experiment affirms what was uncovered in experiment #2 – “page” (the control in this experiment) is not universally beneficial. In fact the usage of “site” resulted in a 93% increase in site creation with a level of confidence of 80%. While the use of “journal” resulted in a 138% increase in site creation with a level of confidence of 91%.
Based on this, it would be beneficial to continue testing the use of “journal” as a potential alternative to “site” or “page”.
Overall, these findings suggest that there is value in exploring alternative language options to reduce perceived commitment or anxiety when prompting users to create a site on CaringBridge.