How fear of loss affects clickthrough rate on an app download popup
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Ended On: 05/26/2020
CaringBridge was using a popup to advertise their mobile app to people who visited their site 5 times. They had proven that a conversational tone and simple design improved clickthrough rate. However, they knew that visitors weren’t just sitting around waiting to download an app—they had to understand what the app would do for them. The control copy had copy about “not missing an update”, but it wasn’t the first sentence. They wanted to test whether leading with fear of loss—”don’t miss an update”—would beat a simple request to try the app.
Research Question
Will introduction of “fear of loss” language increase clickthrough rates?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.5% | ||
| T1: | Treatment #1 | 1.7% | 11.8% | 99.1% |
This experiment has a required sample size of 36,484 in order to be valid. Since the experiment had a total sample size of 139,380, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 11.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
Prioritizing the language regarding “don’t miss out” did appear to strike a chord—increasing clickthroughs by 11.7%. However, there was another variable with this test that was introduced—the treatment used “new and improved” language that the control did not have. The team re-tested this with the “new and improved” language added to the control to understand the impact of that specific variable.