How a campaign-specific “unsubscribe interruptor” affected opt-out rate
CaringBridge
CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.
Experiment Summary
Ended On: 01/11/2022
One of CaringBridge’s internal issues was that people often unsubscribed around their holiday appeal because of the volume of fundraising asks. They wondered if using a “campaign-specific unsubscribe interruptor” would decrease unsubscribe rate.
They created a version of an email that included an offer to “opt out of holiday appeal messaging” for people who “aren’t able to give during this holiday season”. Clicking this link would add people to a suppression list that would only be used for this specific campaign—which would allow them to receive emails in the future and not permanently opt out.
They split the file and monitored unsubscribe rate to determine effectiveness.
Research Question
We believe that using a campaign-specific unsubscribe interruptor for email recipients will achieve a decrease in unsubscribe rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.75% | ||
| T1: | Treatment #1 | 0.65% | -14.2% | 97.1% |
This experiment has a required sample size of 46,703 in order to be valid. Since the experiment had a total sample size of 116,151, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 14.2% decrease in conversion rate
× 0% increase in average gift
Key Learnings
The treatment with the interruptor reduced unsubscribe rate by 14.2%, statistically valid.
This proved to reduce unsubscribe rate, which, for CaringBridge, meant retaining hundreds of emails. This experiment also prompted other questions like “did this allow people to opt out who might have given?”
Future appeals will test this concept with a split audience throughout the entire campaign to see if overall fundraising results are affected.