NextAfter

How an “Email Pause Button” affects unsubscribes

NextAfter

Experiment Summary

Timeframe: 9/11/2020 - 9/14/2020

We often have people request to take a break from our marketing emails for a time. This is normally because they are in the midst of a busy season or they just accessed a resource that gave them tons of ideas and want to focus on those for a bit.

So we created an “Email Pause Button” to allow users to pause emails for a time instead of unsubscribe.

Version A was a just a normal email. Version B included the pause link near the unsubscribe link.

Research Question

Will adding an option to pause email decrease unsubscribe rates?

Design

C: Regular Unsubscribe
T1: Email Pause Button

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: 1.2%
T1: Email Pause Button 2.3% 90.9% 97.3%

This experiment has a required sample size of 1,100 in order to be valid. Since the experiment had a total sample size of 2,802, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 90.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

Interestingly enough, the option to pause email actually increased unsubscribes by 90% – the opposite of what we had intended. It’s possible that this is because it drew more attention to the unsubscribe option and asked people to make a decision – where normally they may not have even been considering unsubscribing.

However, I don’t think this rules out the pausing emails as an option. It may be more effective to weave it into the unsubscribe process as an alternative option to opting-out.


Experiment Documented by Nathan Hill
Nathan Hill is Marketing Director at NextAfter.

Question about experiment #9744

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.