How shortening an email affects clicks
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 12/10/2014 - 12/12/2014
Nonprofits always ask us: how long should our emails be? There’s no absolute right answer for every organization — but it’s something that every nonprofit should be testing. Since the goal of the email is to get the click, DTS decided to test long copy (with a donation-focused call to action in the email) against an shorter email (that made the reader click to find out more).
Research Question
By placing the content on the landing page and using the email to tease and get the click, can we drive additional traffic to the page which will increase the potential number of donors?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Long Email | 0.39% | ||
| T1: | Short Email | 2.3% | 481.4% | 100.0% |
This experiment has a required sample size of 286 in order to be valid. Since the experiment had a total sample size of 14,883, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 481.4% increase in conversion rate
× 38.1% decrease in average gift
260.2% increase in revenue
Key Learnings
Shorter copy with a soft call-to-action drove 481.4% more visitors to the site. However, both treatments had identical response rates which suggests the need to optimize future long-copy landing pages.