CaringBridge

How clarity affects clickthrough rate

Experiment ID: #223

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 11/20/2014 - 11/25/2014

CaringBridge had a donation ask on the homepage that registered users saw when they were logged in. We wanted to increase clickthroughs on this ask to generate more donations. We created a treatment that was highly donor-centric — it gave them a virtual “pat on the back” and reminded them of all the ways that they help those they care about.

Research Question

Does donor-centric language and a more personal connection increase the number of clicks compared to more succinct copy?

Design

C: Short Copy
T1: Long Copy

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Short Copy 0.12%
T1: Long Copy 0.06%-51.7% 100.0%

This experiment has a required sample size of 17,772 in order to be valid. Since the experiment had a total sample size of 312,436, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    51.7% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment delivered 51.7% fewer clicks than the control. This says a lot about not only the audience viewing this, but the context in which the copy appears. It tells us that this audience values clarity over persuasion — a quick, short ask trumps a longer, more personal ask that attempts to draw in their emotions.

It also suggests that this page is highly functional – since it is a “dashboard” of sorts, people use it to get from one place to another, which suggests that future copy should be concise and to the point.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #223

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.