How additional cultivation impacts digital engagement
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 04/12/2019 - 10/12/2019
During a data analysis performed for ADF at the beginning of the calendar year, we discovered a handful of trends related to the email file that were concerning:
- 3 out of 4 ADF subscribers had not opened an email within the last six months
- New and existing donors were twice as likely to disengage as non-donor subscribers
- ADF was sending twice as many solicitation emails as it was cultivation and stewardship-focused emails. Simply put: the subscribers were being asked twice as much as they were given back to.
While the digital fundraising program had been growing, these trends illustrated a potential problem in the near future if they were not corrected. The historical success of ADF’s online fundraising was closely correlated to the size and engagement of the email file.
Given the ambitious fundraising goals, we couldn’t reduce the number of solicitations. So, instead, we decided to test out an additional cultivation series that would increase the number of emails sent to the file. The additional emails would focus on educating and feeding back into the various segments. There would be no additional solicitations.
At a tactical level, this email series consisted of simply-designed emails sent each Friday by one of the authors on the blog (Sarah K). They would include a link to a blog post or a video and we emphasized getting replies to the email.
To validate the impact of this series, we split the email file in half, paying special attention to ensure that there was equal representation of active, lapsed and non-donors and that the average revenue donor was similar between the splits.
Research Question
Will additional cultivation-focused emails increase the overall engagement (email open and click) for the ADF subscribers?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Additional Cultivation | 24.5% | ||
| T1: | Additional Cultivation Emails | 37.3% | 52.2% | 100.0% |
This experiment has a required sample size of 100 in order to be valid. Since the experiment had a total sample size of 132,698, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
52.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running for 6 months, we observed a 52% increase in the likelihood for a subscriber to engage with digital communications. (An engagement is considered an open or click.) We also observed a 41% increase in online revenue from existing donors. When we dug into the unique segments, we observed the following:
- Non-donors saw a 65% increase in their engagement. In the last 3 months of the experiment, this fell to a 51% increase when compared to the control segment.
- Lapsed donors saw a 50% increase in their engagement. In the last 3 months of the experiment, this dropped slightly to a 47% increase when compared to the control segment.
- The active donors saw a 28% increase in their engagement. In the last 3 months of the experiment, this dropped slightly to a 27% increase when compared to the control segment.
Based upon all of these trends, we can derive two key takeaways:
- While the overall engagement dipped slightly the longer people were in the program, overall engagement remained high for the segment involved in the cultivation series.
- The lapsed donors and non-donors were the segment that saw the greatest improvement in their engagement.