NextAfter

How the “fear of loss” positioning impacted email signup rates

Experiment ID: #103543

NextAfter

Experiment Summary

Ended On: 08/09/2022

As a part of the marketing campaign to promote the launch of a new online course for NextAfter, we were promoting email signups to participate in our online “training cohort” program (for a guided tour through the course to maximize course consumption and learning retention).

The day before the course was launched, we decided to write a “last chance” (to signup) email and send it the day before the course went live.

In reviewing the copy for the course promotion, we decided to test using the “fear of loss” concept to see if it would drive a higher email signup rate for the training cohort/course.

Research Question

We believe that telling someone they’ll lose something instead of telling them what they’ll get for prospective customers will achieve increase email signup rates.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.75%
T1: Treatment #1 0.93% 24.1% 54.1%

This experiment has a required sample size of 19,650 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

Unfortunately, we did not generate enough results to scientifically validate the lift found in the treatment. With only a 54.1% level of confidence, these results are sadly no more reliable than a “coin flip chance” of producing the same lift percentages shown.

This was due to a much smaller audience size on the email resend, which ultimately just didn’t produce enough samples.

This concept should be retested in the future to see if pairing “exclusive access” and “the loss of that exclusive access” if the reader doesn’t take action can actually substantially lift response rates in the future.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #103543

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.