How a donor-based appeal (instead of a goal-based appeal) affects conversion rate Experiment ID: #11479

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Ended On: 1/27/2019

CaringBridge had a matching gift offer as part of their year-end campaign. They had recently developed a dynamic thermometer that would update as a goal was reached and had leveraged it to increase conversion. However, they wondered if the large goal amount was too vague, and possibly caused too much cognitive friction in the mind of the reader. Did it make the reader wonder if the large goal could be achieved? Did it make the reader do mental math to approximate how much had been raised? Most importantly—did it communicate to the reader that their gift could have an impact?

They decided to run an experiment with a donor-based appeal, breaking their day down into hourly goals and keeping a dynamic countdown of how many donors were needed to meet the hourly goal. Their hypothesis was that smaller goals would bring it more within reach of the donor, and that communicating a goal in terms of how many donations—rather than how much money—was needed would tell the donor how they could materially affect the goal. Notably, the treatment did not include the match language at all. 

They launched an A/B test to understand the impact of this treatment. 

Research Question

How will a donor-based appeal (instead of a goal-based appeal) affects conversion rate?

Design

C: Thermometer
T1: Power Hour

Results

Treatment Name Click Rate Relative Difference Confidence
C: Thermometer 0.39%
T1: Power Hour 0.49% 25.1% 98.0%

This experiment has a required sample size of 34,742 in order to be valid. Since the experiment had a total sample size of 97,403, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    25.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment version produced a 25% increase in conversion—at scale, a tremendous increase in revenue. This suggested that the donor-based goal was not only easier to understand, but more appealing. This approach will be tested again in future appeals to refine our understanding of this increase. 


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.