How “need-based” copy with a quantified ask affects donor conversion during a matching challenge
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Timeframe: 03/09/2017 - 03/23/2017
CaringBridge had an area on their journals called the “tribute widget”. This widget asked the reader to honor their friend or loved one by making a donation in their honor. This had caused some clarity issues around who was receiving the gift, but testing and optimization had lowered the refund rate significantly.
They were entering a 48-hour matching challenge and wanted to test two new factors in the tribute widget. First, they wanted to make a more direct ask. Rather than asking the reader to “honor” their friend, they wanted to ask them to “help power [their] site”. Additionally, they wanted to include a quantified ask—giving the donor the handle of “$30 powers their site for one month”.
They set up an A/B test during the match period to see how it affected results.
Research Question
Will a quantified ask with a more straightforward appeal increase conversion?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Control | 0.24% | ||
T1: | Quantified Ask | 0.31% | 30.1% | 99.8% |
This experiment has a required sample size of 40,517 in order to be valid. Since the experiment had a total sample size of 193,716, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 30.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment copy in the tribute widget produced a 30.1% increase in donor conversion—a significant increase in conversion. This suggests that CaringBridge hit on some powerful language that hits home with their donors, which inspired them to isolate how much the matching challenge affected results by re-testing this in a normal time period. .
—
Question about experiment #6372
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.