How including a gift handle with the corresponding impact described in a fundraising email affects donor conversion rate
The Navigators
Experiment Summary
Ended On: 04/22/2024
The Navigators are running their bi-annual Eagle Lake Campaign to raise money for this kids’ camp that operates every summer. In the kickoff email of this campaign, they wanted to test how including gift handles in the email would affect donor conversion rate. Each gift amount described what that amount would provide for Eagle Lake.
This experiment idea came from their direct mail piece which included “tickets” for people to purchase. Each ticket was for a different amount (same as the email) and described what each level of ticket would provide for the kids at Eagle Lake.
To keep the language consistent between the email and the donation page, the treatment donation page also had the gift handles in the body copy.
Research Question
We believe that including gift handles with the corresponding impact described for potential donors to Eagle Lake will achieve a higher donor conversion rate because we’re providing clear, tangible examples of how different donation amounts will be used .
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.03% | ||
| T1: | Treatment #1 | 0.02% | -15.1% | 43.7% |
This experiment has a required sample size of 1,066,684 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The key learnings from this experiment are that including gift handles with corresponding impact descriptions in a fundraising email may not always lead to an increase in donor conversion rates. In this case, the treatment actually resulted in a decrease of 15.1% in donations for all traffic. However, since the statistical level of confidence is below 85%, these results cannot be considered valid or conclusive.
In future experimentation, it is important to continue testing different strategies and elements in fundraising emails to see what resonates best with donors. This could include testing different gift handle amounts, impact descriptions, or even the placement of this information within the email. It’s also important to ensure that experiments are conducted with a sufficient sample size to achieve a higher level of statistical confidence in the results.
Overall, the key takeaway is to not solely rely on the results of one experiment, especially if the statistical confidence level is below the recommended threshold. By continuing to test and iterate on different strategies, we can gather more reliable data to inform decision-making for future fundraising campaigns.