How additional cultivation impacts online giving
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 04/12/2019 - 10/12/2019
During a data analysis performed for ADF at the beginning of the calendar year, we discovered a handful of trends related to the email file that were concerning:
- 3 out of 4 ADF subscribers had not opened an email within the last six months
- New and existing donors were twice as likely to disengage as non-donor subscribers
- ADF was sending twice as many solicitation emails as it was cultivation and stewardship-focused emails. Simply put: the subscribers were being asked twice as much as they were given back to.
While the digital fundraising program had been growing, these trends illustrated a potential problem in the near future if they were not corrected. The historical success of ADF’s online fundraising was closely correlated to the size and engagement of the email file.
Given the ambitious fundraising goals, we couldn’t reduce the number of solicitations. So, instead, we decided to test out an additional cultivation series that would increase the number of emails sent to the file. The additional emails would focus on educating and feeding back into the various segments. There would be no additional solicitations.
At a tactical level, this email series consisted of simply-designed emails sent each Friday by one of the authors on the blog (Sarah K). They would include a link to a blog post or a video and we emphasized getting replies to the email.
To validate the impact of this series, we split the email file in half, paying special attention to ensure that there was equal representation of active, lapsed and non-donors and that the average revenue donor was similar between the splits.
Research Question
Will an increase in cultivation emails have any impact on a donor’s likelihood to give?
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | No Additional Cultivation | $9.61 | $179.77 | ||
| T1: | Additional Cultivation Group | $13.60 | 41.5% | 99.9% | $225.00 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 13.1% increase in conversion rate
× 25.2% increase in average gift
41.5% increase in revenue
Key Learnings
After running for a full six months, we observed a statistically significant 13.1% increase in the likelihood for a donor to give a gift online. Combined with the 25% increase in average gift, this resulted in a 41.5% increase in online revenue from the donors.
If this cultivation series were rolled out to the entire file and run for an entire year, the projected impact would be an additional $262K raised online without sending any additional solicitations. We also observed a 43% increase in online engagement for all segments.
While the emphasis of this experiment was to measure the impact of online giving and engagement, we also observed several other trends:
- The impact of the cultivation did not fade over time. We observed the same statistically significant increase in likelihood to give and revenue when we compared the first three months of giving to the second three months.
- The additional cultivation did not significantly impact a donor’s likelihood to give across all channels but we did see an increase in the amount they gave by 30% regardless of channel (with a 99% level of confidence.) It should be noted that this lift did not reach the required sample size so this should be considered a directional lift.
- The additional cultivation did not increase the likelihood of a non-donor to make their first gift. However, it did increase their average revenue per donor by 50% in the 6 month period.
* For all of the statistics above, we excluded both gifts above $10K and all recurring gifts. When we included these additional classifications of gifts, we observed a 14% increase in revenue for a total of $89.7K additional revenue for the treatment segment.