How reducing decision friction impacted email clicks
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Ended On: 04/22/2020
As part of a weekly cultivation series to a segment of their housefile, The Heritage Foundation was sending an invitation that included several links to participate in upcoming webinars taking place on various dates and covering various topics. They wanted to test if sending a single link to an individual webinar would increase email engagement. Our hypothesis was that eliminating the additional webinar options and focusing the call to action on one specific event would optimize for more clicks and lead to additional traffic to the website.
Research Question
Will reducing decision friction by minimizing the number of links included in the email increase click rate?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control - Multiple Links | 0.33% | ||
| T1: | Treatment - Individual Link | 0.43% | 31.5% | 100.0% |
This experiment has a required sample size of 27,186 in order to be valid. Since the experiment had a total sample size of 315,774, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
31.5% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
We saw a 31.5% increase in email clicks by removing the additional webinar links in the email. This indicates that we can drive more traffic to our website from email marketing when using a single call to action with a specific focus rather than presenting additional links to various content. To increase email engagement and website traffic, The Heritage Foundation should continue using this approach for future webinar invitations.