How communal language affects donor conversion for mobile devices
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Ended On: 04/20/2021
CaringBridge’s main donation page had always had a high conversion rate, which made it difficult to produce a winning test. However, they had been learning about communal language and wanted to test it to see if connecting with the prospective donor on a non-transactional level could increase conversion rate.
To instill communal language, they made three radical changes.
First, they changed the headline to language that emphasized the impact that the donor could have, rather than transactional language. Then, they changed the introduction to simply thank the reader for staying connected through CaringBridge and inserted additional examples of the connectedness that the donor’s support provides, on a personal level.
Finally, they increased the presence of “autonomy” by instituting language like “if it’s not too much to ask”, and “if you’re willing”. This language changes a “hard ask” like “Can you support CaringBridge today” to something that puts the power in the donor’s hands, and makes it more of a micro-decision to first consider giving, before deciding to give, and how much to give.
They split traffic to the two pages to determine a winner.
Research Question
We believe that instituting communal language for donation page visitors will achieve an increase in conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 7.3% | ||
| T1: | Treatment #1 | 11.2% | 52.5% | 96.3% |
This experiment has a required sample size of 437 in order to be valid. Since the experiment had a total sample size of 985, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 52.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment with the communal language did not appear to increase conversion overall. However, when mobile devices were isolated as a segment, it emerged that there was a 52.5% increase in conversion, with 96.5% validity.
The hypothesis is that on mobile devices, absorption of the value proposition copy is more forced as the smaller screen requires engagement, whereas on a larger desktop device, it’s easier to skip over the copy as it takes up much less real estate on the page.
There are two important learnings from this experiment. First, that communal language can have a positive impact on donor conversion, and second, that sometimes important increases and learnings are lurking beneath the surface in segment-level data.