How presenting a daily goal in an email appeal affects donations during a year-end fundraising campaign
American Cornerstone Institute
Experiment Summary
Timeframe: 11/29/2022 - 12/27/2022
For this experiment, we wanted to test how presenting a daily goal (“today I’m looking for X donors”) in the body copy of an email appeal would affect donation rates for a public policy organization during a year-end fundraising campaign.
For this experiment, both previous donors and non-donors were included in the sample universe with ~50% of appeal recipients receiving the control (containing an ask that did not include a daily goal) and ~50% of appeal recipients receiving the treatment (containing an ask centered around a daily goal. Nothing else about the appeal was changed.
Major donors were excluded from this test and our experiment spanned seven appeal emails.
Research Question
We believe that including a daily goal in the body of an email appeal will increase donation rates by increasing the specificity of the ask and giving recipients a clear ‘micro’ goal that they can help achieve.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 9.4% | ||
| T1: | Treatment #1 | 8.3% | -12.2% | 84.0% |
This experiment has a required sample size of 4,686 in order to be valid. Since the experiment had a total sample size of 4,824, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
After running this experiment over the course of seven emails during the organization’s year-end fundraising campaign, we observed that the treatment produced a 12.2% decrease in donations (84% level of confidence).
The sample size of this experiment did not allow us to validate these results and therefore prevents us from drawing statistically valid conclusions about how presenting a daily goal in the body of an appeal email affects donation rates.
However, the results of this experiment do suggest that including a daily goal was not a motivating factor for recipients of this appeal and may even have demotivated them from giving. We also observed that the daily goal did not serve as a significant motivating factor in terms of clicking through to the donation page from the email, producing only a 2.1% lift with a 53.6% level of statistical confidence.
This may be because a daily goal does not add tangible value or a reason to give to the appeal. It is also in essence an organizational-focused goal that lacks a specific outcome that the daily goal will help achieve.
Based on these results, this public policy client may want to pursue a different method for increasing engagement and donations on the next high-urgency campaign or consider including a specific outcome that the daily goal will achieve. In either case, the organization should not assume that including a daily goal, as structured in this experiment, will increase donation rates during a high-urgency campaign.