How relevant, non-specific “crisis” language affects conversion

Experiment ID: #20513


CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 03/19/2020 - 04/01/2020

When COVID-19 became a worldwide pandemic, the importance of CaringBridge became greater. As people were sheltering-in-place at their homes, the digital connections CaringBridge helped facilitate were more significant—which also increased the urgency to support CaringBridge as a platform. They wanted to find the right way to integrate this into their fundraising message, hoping that it would increase conversion rate. They created messaging that didn’t address COVID-19 specifically, as not to be exploitative, but addressed the larger sentiment. Then, they tested this in their highest revenue placement, the tribute widget.

Research Question

How will relevant, non-specific “crisis” language affects conversion?


C: Control
T1: Times Like These


 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.18%
T1: Times Like These 0.19%2.7% 42.6%

This experiment has a required sample size of 5,838,897 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

After seeing an initial increase in conversion (that got as high as 15% in the first week), the data normalized at no significant increase in response. This could be because the copy was NOT specific to COVID-19, but could also reflect the fact that prospective donors did not share the sentiment that CaringBridge becomes more valuable in crisis times. There are two possible paths to take here:

  1. Continue to test non-time-sensitive copy on the tribute widget
  2. Be more explicit in the COVID-19 messaging to see if response can be increased with more specificity.

Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #20513

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.