How adding 3 links to a fundraising email for non-members affects the click-through rate.
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 03/01/2024 - 03/01/2024
We conducted this email experiment for The Heritage Foundation’s Founder’s Day campaign. In the email, the President thanked non-members for their generosity and the issues America is facing. The conclusion of the email includes a call to action to donate.
We wanted to know if having three hyperlinked calls to action placed at the beginning, middle, and end of the email would encourage non-members to click more than having only one call to action at the end.
Research Question
We believe that having three calls to action throughout an email for non-members will achieve a higher click-through rate and lead to an increase in donations because it provides more opportunity to take action.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.18% | ||
| T1: | Treatment #1 | 0.35% | 100.2% | 100.0% |
This experiment has a required sample size of 6,499 in order to be valid. Since the experiment had a total sample size of 759,462, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
100.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that having three calls to action throughout an email for non-members resulted in a significantly higher click-through rate compared to having only one call to action at the end. The treatment group, which had three hyperlinked calls to action placed at the beginning, middle, and end of the email, saw a 100.2% increase in clicks for all traffic with a 100.0% statistical level of confidence.
Note: While the click-through rate increased the donation conversion rate was the same for both the control and treatment email.
In future experimentation, this learning should be applied by incorporating multiple calls to action strategically throughout emails or communication pieces. By providing more opportunities for recipients to take action, it increases the likelihood of engagement and conversion. Additionally, testing different placements and wording for these calls to action could further optimize the approach to maximize results. This experiment underscores the importance of experimentation and testing in marketing strategies to identify the most effective tactics for achieving goals.