CaringBridge

How “minor mystery” language in a headline affects conversion

Experiment ID: #8455

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 01/03/2018 - 01/14/2018

CaringBridge had run extensive testing on their “tribute widget”, where the majority of donations on their site begin. They wanted to run a headline test, with the knowledge that such a slight change might not produce any noticeable effect. They wanted to test the angle of a “minor mystery”, which had proven in other experiments to increase engagement (and subsequently conversion) by provoking the reader that they might not know something.

The headline on the tribute widget asked a simple question: Can you help power [tribute name]’s site? They proposed a test to the headline to introduce the “minor mystery”: Did you know you can help power [tribute name]’s site?

Research Question

How will “minor mystery” language in a headline affect conversion?

Design

C: Control
T1: Did You Know

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 0.09%$52.74
T1: Did You Know 0.09%-1.4% 24.8%$51.00

This experiment has a required sample size of 39,600,034 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

There was no significant difference in the two headlines. This means that the next experiment needs to be more radical in nature to affect results.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #8455

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.