CaringBridge

How a higher suggested ask amount affects conversion and revenue

Experiment ID: #56441

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 04/02/2021 - 04/23/2021

CaringBridge’s tribute widget had a $30 ask, which they had calculated as the amount of money that it takes to keep one site running for a full month. Interestingly, the amount wasn’t reflected on the following donation page — and when that amount was added, it reduced conversion. They wanted to test the opposite—bringing the suggested amount from the following donation page into the tribute widget. To do this, they suggested an ask of $50. They hypothesized that this might decrease conversion, but that they might make up the decrease in additional average gift.

Research Question

We believe that increasing the ask amount for journal visitors will achieve an increase in revenue.

Design

C: Control
T1: $50 Ask

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 0.37%$0.00
T1: $50 Ask 0.32%-11.8% 99.8%$0.00

This experiment has a required sample size of 140,720 in order to be valid. Since the experiment had a total sample size of 709,334, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 11.8% decrease in conversion rate
× 0% increase in average gift

Key Learnings

This increased suggested ask produced an 11.8% decrease in conversion rate. This might be acceptable, but the increase in average gift was only $4, which did not offset the revenue lost from the reduction in donations.

The ideal outcome would have been a more marginal decrease in conversion rate, as more revenue is not always an even trade for more donors. When you get more donors, you have more people you can renew or upgrade. However, donors who give at higher amounts are often able to give much more over their lifetime with the organization.

This is a perfect example of why we test—to discover risks and tradeoffs without making drastic changes.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #56441

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.