How a more direct, urgent and exclusive positioning of an offer impacted email signup rates
NextAfter
Experiment Summary
Ended On: 08/03/2022
As a part of promoting a new course, the team at NextAfter was going to send a special invitation email to subscribers to invite them to participate in an online training cohort through its NextAfter Institute.
In reviewing the control email invitation — the team focused on how a “radical redesign” treatment may increase demand and ultimately drive a higher number of cohort signups.
In reviewing the control, we identified a number of variables that could serve as a testing treatment — but with limited time to get the email out on time, we decided to hit on all variables that we isolated.
Specifically, we identified 6 variables that could be tested, and the treatment addressed every one, which are:
- Anxiety — defined as “psychological resistance” for the audience to take the presented action. In our control, we identified that putting too much information (the training schedule) could lead less people to sign up.
- Brevity/Directness —the length of the email was long, and much of the messaging wasn’t necessary for the reader to determine whether or not they wanted to signup for the cohort. So, we theorized that shortening the message and being more direct could drive a higher signup rate.
- Anticipating Objections — We thought through why someone would decline to participate in the training cohort. Our top reason is that the training schedule didn’t permit them from participating in the live training sessions (on-time), so we wanted to address that in the email message.
- Urgency — Both treatments talked about the upcoming start date for the training cohort that we were asking them to sign up for, but we wanted to increase the urgency that someone had to take action right now.
- Exclusivity — We wanted the reader to feel special that they were being personally invited, and a part of group that would have first access to the new course/training cohort.
- Intrigue — We wanted to pique the reader’s curiosity and leave some of the control email (the schedule) behind the click. By giving away too much in the inbox, we could prematurely force our audience to make a rash decision against signing up for the course. Therefore, we wanted to tease out more clicks and allow the strength of the landing page (and the cognitive momentum of them clicking to “learn more” about the course) to carry them through the registration process for the training cohort.
As noted, because there were too many variables to isolate and too little time to reasonably test successively each concept, we decided to develop a treatment to drive all of these elements into the treatment message shown below.
Research Question
We believe that being more direct, urgent, and exclusive in our invitation for subscribers and members at NextAfter will achieve increase email signups.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.63% | ||
| T1: | Treatment #1 | 1.2% | 96.4% | 100.0% |
This experiment has a required sample size of 1,933 in order to be valid. Since the experiment had a total sample size of 35,653, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 96.4% increase in conversion rate
× 0% increase in average gift
Key Learnings
With a 99.9% level of confidence, we increased email signups for the new course (and training cohort) by +96.4%.
What’s interesting is that we saw far more clicks (+76%, with a confidence level of 100%) on the treatment email, as well. There have been cases where if you pull back some information (through the “intrigue” concept), you can drive more clicks, but with a lower post-click conversion rate on the subsequent landing page.
This was not the case for the treatment email. In fact, it drove more clicks, and the landing page signup rate was actually validated (after the click) to increase email signups on the same landing page by +28.5% (with a confidence score of 99.6%).
This means that we didn’t drive “exploratory clicks” from the treatment email, but rather that the clicks we drove from the treatment were more motivated to signup for the course than those clicking from the control email were.
Each of these variables should be isolated in subsequent testing to determine the efficacy of increasing motivation for each component used within the treatment message, but suffice it to say that you should consider each of these variables when attempting to drive a motivated click that is more likely to respond to your message in the way that you want.