How directly asking the visitor to give with a question affects conversion rate Experiment ID: #19923

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 2/10/2020 - 2/15/2020

CaringBridge received most of their donor revenue through an area on the site called the tribute widget. This tribute widget asked visitors to make a donation in honor of their friend who they were visiting, and then gave them the opportunity to write a message along with their donation. Over the years, this tribute widget had been tested repeatedly to find the ideal combination of language that would motivate donors to give. Upon reviewing the most recent control, the CaringBridge team noticed that the call to action was a command, rather than a question: “Help keep this site online for them and for you”. They were curious to find out if asking a question convert more people than a command. So they drafted a new treatment that asked the visitor: “Will you make a gift to ensure that this site stays online for them and for you?” It was a very slight change to the copy, but a very significant change in how it addressed the visitor. They launched an experiment to see the effect on donor conversion.

Research Question

How will directly asking the visitor with a question to give affect conversion rate?

Design

C: Control
T1: With question

Results

Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.08% $0.00
T1: With question 0.11% 33.0% 98.5% $0.00

This experiment has a required sample size of 102,969 in order to be valid. Since the experiment had a total sample size of 314,853, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 33.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment, which asked the user a question, produced a 33% increase in donor conversion. This was primarily due to two factors: first, the treatment made it clear that something was being asked of the visitor by placing the call to action in the form of a question. Second, the treatment quantified what it was asking the visitor to do: make a donation.

This combination of language can be tested in other areas of the site. However, it must be noted that in lower conversion areas, this may have diminishing results as users may not be properly motivated.


Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.