How directly asking the visitor to give with a question affects conversion rate
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Timeframe: 02/10/2020 - 02/15/2020
CaringBridge received most of their donor revenue through an area on the site called the tribute widget. This tribute widget asked visitors to make a donation in honor of their friend who they were visiting, and then gave them the opportunity to write a message along with their donation. Over the years, this tribute widget had been tested repeatedly to find the ideal combination of language that would motivate donors to give. Upon reviewing the most recent control, the CaringBridge team noticed that the call to action was a command, rather than a question: “Help keep this site online for them and for you”. They were curious to find out if asking a question convert more people than a command. So they drafted a new treatment that asked the visitor: “Will you make a gift to ensure that this site stays online for them and for you?” It was a very slight change to the copy, but a very significant change in how it addressed the visitor. They launched an experiment to see the effect on donor conversion.
Research Question
How will directly asking the visitor with a question to give affect conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.08% | |||
| T1: | With question | 0.11% | 33.0% | 98.5% |
This experiment has a required sample size of 102,969 in order to be valid. Since the experiment had a total sample size of 314,853, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 33.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment, which asked the user a question, produced a 33% increase in donor conversion. This was primarily due to two factors: first, the treatment made it clear that something was being asked of the visitor by placing the call to action in the form of a question. Second, the treatment quantified what it was asking the visitor to do: make a donation.
This combination of language can be tested in other areas of the site. However, it must be noted that in lower conversion areas, this may have diminishing results as users may not be properly motivated.