How pre-checking an email opt-in box affects email opt-ins
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Ended On: 03/23/2020
As part of their new user registration process, CaringBridge had always given people the option to opt-in to marketing emails, new content, stories, and helpful articles as they created their account. However, that check box had always been unchecked.
Research Question
How will pre-checking an email opt-in box affect email opt-in rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 8.1% | ||
| T1: | Treatment #1 | 18.3% | 126.7% | 100.0% |
This experiment has a required sample size of 84 in order to be valid. Since the experiment had a total sample size of 90,341, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 126.7% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment, with the pre-checked box, resulted in a 126.7% increase in email address acquisition rate. This showed that the small act of checking a box was a tremendous point of friction to users, who might otherwise be amenable to opting in. This showed that the cost of checking the box was too high—which had nothing to do with the value (or perceived value) of the subscription. Once the cost was taken care of, the user was much more likely to accept the value.