How “minor mystery” language in a headline affects conversion Experiment ID: #8455

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 1/3/2018 - 1/14/2018

CaringBridge had run extensive testing on their “tribute widget”, where the majority of donations on their site begin. They wanted to run a headline test, with the knowledge that such a slight change might not produce any noticeable effect. They wanted to test the angle of a “minor mystery”, which had proven in other experiments to increase engagement (and subsequently conversion) by provoking the reader that they might not know something.

The headline on the tribute widget asked a simple question: Can you help power [tribute name]’s site? They proposed a test to the headline to introduce the “minor mystery”: Did you know you can help power [tribute name]’s site?

Research Question

How will “minor mystery” language in a headline affect conversion?

Design

C: Control
T1: Did You Know

Results

Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.09% $52.74
T1: Did You Know 0.09% -1.4% 24.8% $51.00

This experiment has a required sample size of 39,600,034 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

There was no significant difference in the two headlines. This means that the next experiment needs to be more radical in nature to affect results.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.