How a campaign-specific "unsubscribe interruptor" affected opt-out rate - NextAfter
CaringBridge

How a campaign-specific “unsubscribe interruptor” affected opt-out rate

Experiment ID: #81890

CaringBridge

CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Ended On: 01/11/2022

One of CaringBridge’s internal issues was that people often unsubscribed around their holiday appeal because of the volume of fundraising asks. They wondered if using a “campaign-specific unsubscribe interruptor” would decrease unsubscribe rate.

They created a version of an email that included an offer to “opt out of holiday appeal messaging” for people who “aren’t able to give during this holiday season”. Clicking this link would add people to a suppression list that would only be used for this specific campaign—which would allow them to receive emails in the future and not permanently opt out.

They split the file and monitored unsubscribe rate to determine effectiveness.

Research Question

We believe that using a campaign-specific unsubscribe interruptor for email recipients will achieve a decrease in unsubscribe rate.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.75%
T1: Treatment #1 0.65% -14.2% 97.1%

This experiment has a required sample size of 46,703 in order to be valid. Since the experiment had a total sample size of 116,151, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 14.2% decrease in conversion rate
× 0% increase in average gift

Key Learnings

The treatment with the interruptor reduced unsubscribe rate by 14.2%, statistically valid.

This proved to reduce unsubscribe rate, which, for CaringBridge, meant retaining hundreds of emails. This experiment also prompted other questions like “did this allow people to opt out who might have given?”

Future appeals will test this concept with a split audience throughout the entire campaign to see if overall fundraising results are affected.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #81890

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.