How simplifying the email design affects clicks
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 12/20/2014 - 12/22/2014
The design of the email did not have any significant impact on the supporter’s likelihood of clicking on the email. It also did not have any significant impact on the donors’ propensity to give. Moving forward we will utilize the existing email templates to save both cost and time.
Research Question
A simplified email will have a more personal feel to it which will prompt a greater number of click throughs.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Designed Email | 7.5% | ||
| T1: | Simplified Email | 7.3% | -3.1% | 40.7% |
This experiment has a required sample size of 99,515 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The standard format for DTS emails is a designed layout with images and a color scheme that matches the website. In an effort to increase the number of visitors to the calendar year end campaign, we created an experiment to test the standard design against a simplified email.