How a more human subject line affects conversion rates
NextAfter
Experiment Summary
Timeframe: 08/10/2020 - 08/11/2020
In this experiment, we were launching a free training course to help nonprofits improve their copywriting. The initial copy included a subject line that utilized sales-style tactics to try and win the open. But we wondered if a more humanized version of the subject line could lead to more opens – and possibly even more registrations.
Research Question
Will a more human sounding subject line lead to more registrations?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Free: [Limited Time] Nonprofit Copywriting “Faster Class” | 2.5% | ||
| T1: | A free copywriting training class for you | 3.6% | 41.9% | 100.0% |
This experiment has a required sample size of 2,040 in order to be valid. Since the experiment had a total sample size of 13,657, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
41.9% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The new subject line led to a statistically valid increase in a) Open, b) Clicks, and c) Course Registrations. In fact, the subject line made the most significant impact on registrations, leading to a 41.9% increase in people registering for the free training.
This experiment is another solid indication that making your subject line sound more believable, human, and empathetic impacts the perception of the entire email – as well as the landing page.