How increasing clarity in the value proposition and creating differentiation in the call-to-action affects new site signups Experiment ID: #16858
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Timeframe: 6/14/2019 - 7/1/2019
CaringBridge wanted to increase the number of new journal sites that were created from the homepage, primarily from mobile devices (which constituted the majority of traffic). After an initial review of the homepage, a few potential issues were identified. First, new site signups were not prioritized—”visiting a friend” was above it in the visual hierarchy. On mobile, the headline “Start your free website” and the call to action “Start a site” were right next to each other, with no description in between. In fact, most of the value proposition was actually below the call to action. They decided to create two treatments—
The first treatment used a visual image of someone using a computer to access CaringBridge—to reinforce the feeling that a CaringBridge site creates. Second, a new call-to-action was used: “Create a site”, instead of “Start a site”, to emphasize that the site was new. Finally, the headline was changed to clarify what CaringBridge offered—”Personal, protected websites for every health journey”. This copy was on the treatment, but was secondary to the headline “Share, Connect and Rally Support”.
They wanted to also see if “help text” would further increase conversions. They knew this would vary based on the visitor’s familiarity with CaringBridge, but they added additional bullet points of copy, as well as personalizing the headline to reflect what CaringBridge can do for you, the visitor: “Start your free, personal CaringBridge site today”.
They launched a three-way test to determine which experience drove the most new site signups from the homepage.
Will increasing the clarity of the value proposition and differentiating the CTAs increase new site signups?
|Treatment Name||Conv. Rate||Relative Difference||Confidence||Average Gift|
|T1:||Short Text - Card||0.57%||31.1%||99.6%||$0.00|
|T2:||Long Text - Card||0.48%||10.0%||57.2%||$0.00|
This experiment has a required sample size of 31,092 in order to be valid. Since the experiment had a total sample size of 113,400, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 31.1% increase in conversion rate
× 0% increase in average gift
The first treatment, with the shorter text, delivered a 31% increase to new site signups from mobile traffic. Overall, this treatment increased signups by 7.8%, but the overall results were not statistically valid.