How including a story of impact in a calendar year-end campaign affects donor conversion
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Ended On: 01/11/2022
CaringBridge had used lots of stories in fundraising appeals before. But one of their challenges is that the value proposition of CaringBridge is largely built through personal experience—your ability to interact with a friend or loved one—and, thus, third-party stories sometimes fell flat. However, they wanted to do more testing to understand how usage of a story could potentially impact the results of an appeal.
They saw an opportunity in their holiday appeal to test the concept by inserting a story into the middle of an otherwise straightforward match appeal. They positioned the story so that it came as a “personally impactful story”, not just a random case study on the impact of CaringBridge.
They split the email file so only half saw the story of impact and tested the concept.
Research Question
We believe that including a story of impact for prospective donors will achieve an increase in conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.11% | ||
| T1: | Treatment #1 | 0.15% | 42.0% | 95.3% |
This experiment has a required sample size of 48,394 in order to be valid. Since the experiment had a total sample size of 99,372, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 42.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment saw a 42% increase in conversion rate, just over the 95% threshold for a valid level of confidence.
CaringBridge believes the success of this experiment is largely due to the way in which the story was positioned. While this is a great lift, it needs to be tested further to determine if there are diminishing returns.