How adding 3 links to a fundraising email for lapsed members affects the click-through rate.
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Ended On: 03/01/2024
We conducted this email experiment for The Heritage Foundation’s Founder’s Day campaign. In the email, the President thanked lapsed members for their generosity and the issues America is facing. The conclusion of the email includes a call to action to donate.
We wanted to know if having three hyperlinked calls to action placed at the beginning, middle, and end of the email would cause lapsed members to click more than having only one call to action at the end.
Research Question
We believe that having three calls to action throughout an email for lapsed members will achieve a higher click-through rate and lead to an increase in donations because it provides more opportunity to take action.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.13% | ||
| T1: | Treatment #1 | 0.44% | 242.7% | 100.0% |
This experiment has a required sample size of 2,220 in order to be valid. Since the experiment had a total sample size of 60,189, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
242.7% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that having three calls to action throughout an email for lapsed members significantly increased the click-through rate but did not lead to an increase in donations. The treatment group, which had three hyperlinked calls to action at the beginning, middle, and end of the email, resulted in a 242.7% increase in clicks for all traffic compared to the control group, which only had one call to action at the end of the email.
Note: While the click-through rate increased the donation conversion rate decreased. The sample size was not large enough to attain the required level of confidence of 95% or above.
This shows that providing more opportunities for recipients to take action within the email can greatly impact their engagement and willingness to donate. In future experimentation, it would be beneficial to continue testing the placement and frequency of calls to action in emails to optimize engagement and donation rates. Additionally, it is important to always consider the preferences and behaviors of the target audience when designing email campaigns.