How increasing personalization and aligning motivation with user intent affects engagement Experiment ID: #8437

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 1/16/2018 - 1/31/2018

CaringBridge’s marketing team sought to increase the amount of users who got to their “ways to help” page, where there were several options to deepen engagement and provide more support to their friend or family member.

They thought that a slight copy change would align the “Ways to help” button on the journal page with the user’s intent. Instead of “Ways to help”, they proposed “Help [FirstName]” to encourage more people to click through. They thought this might work because they hypothesized that people who were reading this page were (consciously or unconsciously) desiring to help their friend or family member.

They were concerned about possible reactions to this test, so they only included 10% of site traffic in the experiment.

Research Question

Will increasing personalization and aligning motivation with user intent increase engagement?

Design

C: Control
T1: Treatment 1

Results

Treatment Name Click Rate Relative Difference Confidence
C: Control 0.34%
T1: Treatment 1 0.40% 19.7% 100.0%

This experiment has a required sample size of 61,157 in order to be valid. Since the experiment had a total sample size of 4,557,343, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    19.7% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The new copy produced a 19% lift in clickthrough rate for the “Ways to help” page—an impressive lift considering the small change. This points to more opportunities to align site copy with user intent throughout the site. With the volume of traffic that CaringBridge has, there is tremendous potential to increase clarity and adjust user experience accordingly.


Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.