How an “Email Pause Button” affects unsubscribes
NextAfter
Experiment Summary
Timeframe: 09/11/2020 - 09/14/2020
We often have people request to take a break from our marketing emails for a time. This is normally because they are in the midst of a busy season or they just accessed a resource that gave them tons of ideas and want to focus on those for a bit.
So we created an “Email Pause Button” to allow users to pause emails for a time instead of unsubscribe.
Version A was a just a normal email. Version B included the pause link near the unsubscribe link.
Research Question
Will adding an option to pause email decrease unsubscribe rates?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Regular Unsubscribe | 0.99% | ||
| T1: | Email Pause Button | 2.3% | 131.9% | 99.3% |
This experiment has a required sample size of 725 in order to be valid. Since the experiment had a total sample size of 2,802, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 131.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
Interestingly enough, the option to pause email actually increased unsubscribes by 90% – the opposite of what we had intended. It’s possible that this is because it drew more attention to the unsubscribe option and asked people to make a decision – where normally they may not have even been considering unsubscribing.
However, I don’t think this rules out the pausing emails as an option. It may be more effective to weave it into the unsubscribe process as an alternative option to opting-out.