How “minor mystery” language in a headline affects conversion Experiment ID: #8455
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Timeframe: 1/3/2018 - 1/14/2018
CaringBridge had run extensive testing on their “tribute widget”, where the majority of donations on their site begin. They wanted to run a headline test, with the knowledge that such a slight change might not produce any noticeable effect. They wanted to test the angle of a “minor mystery”, which had proven in other experiments to increase engagement (and subsequently conversion) by provoking the reader that they might not know something.
The headline on the tribute widget asked a simple question: Can you help power [tribute name]’s site? They proposed a test to the headline to introduce the “minor mystery”: Did you know you can help power [tribute name]’s site?
How will “minor mystery” language in a headline affect conversion?
|Treatment Name||Conv. Rate||Relative Difference||Confidence||Average Gift|
|T1:||Did You Know||0.09%||-1.4%||24.8%||$51.00|
This experiment has a required sample size of 39,600,034 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
There was no significant difference in the two headlines. This means that the next experiment needs to be more radical in nature to affect results.