How relevant, non-specific “crisis” language affects conversion

Experiment ID: #21354


CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 3/19/2020 - 4/1/2020

When COVID-19 became a worldwide pandemic, the importance of CaringBridge became greater. As people were sheltering-in-place at their homes, the digital connections CaringBridge helped facilitate were more significant—which also increased the urgency to support CaringBridge as a platform. They wanted to find the right way to integrate this into their fundraising message, hoping that it would increase conversion rate. They created messaging that didn’t address COVID-19 specifically, as not to be exploitative, but addressed the larger sentiment. Then, they tested this in their highest revenue placement, the tribute widget.

Research Question

How will relevant, non-specific “crisis” language affects conversion?


Treatment Name Conv. Rate Relative Difference Confidence
C: 0.18%
T1: Times Like These 0.19% 2.7% 42.6%

This experiment has a required sample size of 5,838,897 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

After seeing an initial increase in conversion (that got as high as 15% in the first week), the data normalized at no significant increase in response. This could be because the copy was NOT specific to COVID-19, but could also reflect the fact that prospective donors did not share the sentiment that CaringBridge becomes more valuable in crisis times. There are two possible paths to take here:

  1. Continue to test non-time-sensitive copy on the tribute widget
  2. Be more explicit in the COVID-19 messaging to see if response can be increased with more specificity.

Experiment Documented by Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.

Question about experiment #6499

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.