How increasing the clarity of the value proposition affects donor conversion rate
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Timeframe: 01/04/2018 - 03/12/2018
JEN (or Journal Email Notification) emails are the lifeblood of the service that CaringBridge provides. These emails are sent out when an author creates a new post in a site, and CaringBridge sends out nearly one million of them each week. There is a donation ask in these emails, but it had very low conversion. CaringBridge wanted to increase revenue from these emails by optimizing the language around the donation ask.
The default language made a statement: Tribute donations are vital to keeping CaringBridge and [FirstName]’s website running. Honor [FirstName] with a donation to CaringBridge. They hypothesized that this copy did not present a clear value proposition to the donor because “tribute donation” is an internal term, and the call-to-action does not specify that the donation that they would honor their friend with is a tribute donation.
They created four treatment versions to run weekly over a monthlong period to see if they could increase conversion rate.
The first treatment didn’t change the copy or CTA, but added a single line question that addressed the context for the ask: “Do you appreciate staying connected to [FirstName] like this?”
The second treatment kept that introductory question, but changed the value proposition of the question to remind the user that CaringBridge was a free service, and asked them to make a generous donation in honor of their friend.
The third treatment quantified the ask with some language that had proven a lift in other areas of the site by telling the user what their gift does, in tangible terms.
Finally, the fourth treatment added a single piece of value proposition to the end of the third treatment—asking the reader to keep their friend’s site online for a year.
CaringBridge launched a weekly testing schedule to determine a winner.
Research Question
Will clarifying the value proposition increase conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.01% | ||
| T1: | Treatment 1 | 0.01% | 7.7% | 39.4% |
| T2: | Treatment 2 | 0.01% | 8.3% | 42.6% |
| T3: | Treatment 3 | 0.02% | 64.4% | 100.0% |
| T4: | Treatment 4 | 0.02% | 78.4% | 100.0% |
This experiment has a required sample size of 408,843 in order to be valid. Since the experiment had a total sample size of 4,662,741, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 64.4% increase in conversion rate
× 0% increase in average gift
Key Learnings
The first two treatments produced marginal lifts that did not reach statistical validity. However, the last two tests produced large, statistically valid lifts with the quantified language and the additional, time-based value proposition.
This shows that clarifying the ask, removing “insider” language, and providing some timeline for the gift significantly increased conversion rate. These tests produced nearly 50 more donations per week—a tremendous amount that will produce large revenue gains over the next year.