Will shortening a promotional email increase click-throughs?

NextAfter

Experiment Summary

Timeframe: 07/21/2023 - 07/24/2023

At NIO Summit, there’s a lot going on. Beyond the in-depth sessions that dig deep into the most recent data and strategies in digital fundraising, there’s a whole theme that we lean into—heavily.

Each month when there’s a ticket deadline it’s often challenging to pick and choose which unique value of NIO to drive forward to the audience that will make them want to come. And often this leads to long emails with a lot of details. 

After seeing a very parsed-down email from Sticker Mule about a deal on t-shirts that they were running, we wondered how that might apply to an upcoming ticket deadline promo we were planning to send to our subscribers. 

We wanted to see if communicating all the many benefits of attending NIO Summit in a single email was creating mental friction. And whether a simpler, more direct email, focused on a single message might eliminate that friction.

Research Question

We believe that shortening a promotional email for our general subscribers will achieve an increase in click-throughs because the email is more direct, human, and will create a higher sense of urgency by focusing exclusively on the upcoming ticket price increase.

Design

C: Version A

T1: Version B

Results

This experiment has a required sample size of 2,241 in order to be valid. Since the experiment had a total sample size of 16,621, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    138.4% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

While both emails had about a 20% open rate, the treatment email resulted in a 138% increase in clicks. Beyond an increase in clicks, it’s important to note that 3 people who received the treatment email bought tickets to NIO Summit, while no tickets were bought by people who received the control email. 

This supports our hypothesis that our typical value-heavy promotional emails may be seen as cluttered and create mental friction. And that the shortened promotional email may increase click-throughs by creating more clarity and a higher sense of urgency.

We plan to test this same hypothesis in other contexts. Our next iteration will be to test this with an email releasing a new online fundraising resource.

Winston’s Analysis:

The key learning from this experiment is that shortening a promotional email to focus on a single message can lead to a significant increase in click-through rates. The treatment email, which was more direct and focused exclusively on the upcoming ticket price increase, resulted in a 138.4% increase in clicks for all traffic.

This suggests that providing a clear and urgent call-to-action, without overwhelming the audience with too much information, can be more effective in motivating subscribers to take action.

Furthermore, the experiment also revealed a 591% decrease in Clicks to Opens. This indicates that the shorter email may have created a higher sense of urgency, resulting in subscribers clicking on the email without spending much time engaging with its content.

In future experimentation, it would be beneficial to continue exploring the impact of shorter, more focused emails on click-through rates. This can be applied not only to promotions for ticket deadlines but also to other types of email communications.

Additionally, it would be valuable to analyze the conversion rates and overall engagement of subscribers who clicked on the shorter email versus the longer email. This will help determine the quality of the clicks and the effectiveness of the treatment in driving desired actions.

Overall, the key takeaway is that simplicity and directness in email messaging can lead to higher engagement and click-through rates. Communicating a single clear message that creates a sense of urgency can be a powerful strategy in future email promotions.