CaringBridge

How “need-based” copy with a quantified ask affects donor conversion during a matching challenge

Experiment ID: #6372

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 03/09/2017 - 03/23/2017

CaringBridge had an area on their journals called the “tribute widget”. This widget asked the reader to honor their friend or loved one by making a donation in their honor. This had caused some clarity issues around who was receiving the gift, but testing and optimization had lowered the refund rate significantly.

They were entering a 48-hour matching challenge and wanted to test two new factors in the tribute widget. First, they wanted to make a more direct ask. Rather than asking the reader to “honor” their friend, they wanted to ask them to “help power [their] site”. Additionally, they wanted to include a quantified ask—giving the donor the handle of “$30 powers their site for one month”.

They set up an A/B test during the match period to see how it affected results.

Research Question

Will a quantified ask with a more straightforward appeal increase conversion?

Design

C: Control
T1: Quantified Ask

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.24%
T1: Quantified Ask 0.31%30.1% 99.8%

This experiment has a required sample size of 40,517 in order to be valid. Since the experiment had a total sample size of 193,716, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 30.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment copy in the tribute widget produced a 30.1% increase in donor conversion—a significant increase in conversion. This suggests that CaringBridge hit on some powerful language that hits home with their donors, which inspired them to isolate how much the matching challenge affected results by re-testing this in a normal time period. .


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #6372

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.