How a more personal tone in a direct mail letter can impact donor conversion
Back to the Bible
Experiment Summary
Timeframe: 03/03/2020 - 04/16/2020
Back to the Bible has shifted the tone and approach to their digital communication dramatically in the past year. The emails they send to people are very conversational, intentional, and personal. Their email file health has seen a dramatic improvement since this change was put in place. The focus of the emails has been to care for people and their spiritual life. Because of this responses digitally, we were curious how this approach might affect their Direct Mail strategy. To test this, we wrote two different Direct Mail letters. One letter was more traditional in nature – the call-to-action focused on the premium offer and included a P.S. at the end of the letter. The treatment was more conversational, soft, and personal in nature – more similar to the approach we’ve taken in their digital communication. We split their Direct Mail file in half and made sure there was an equal number of donors and nondonors for each variable.
Research Question
Will a more personal approach to Direct Mail impact donor conversion and overall revenue?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Traditional Direct Mail | 3.2% | ||
| T1: | More Personal Direct Mail | 3.0% | -4.5% | 33.6% |
This experiment has a required sample size of 113,858 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
After six weeks of the Direct Mail piece being in homes, we pulled the results of the experiment and found that there was no significant difference in donor conversion. We did see more overall revenue from the treatment, but this is mostly attributed to a handful of major gifts that came in from that variable. We will continue to monitor the results of this campaign over the next few weeks, but we’d also like to test this approach again and take an even more radical approach between the two letters. As you can see in the copy, the main body of the text was the same between the two versions.
Next, we’d like to test taking the digital appeal and just repurpose it for Direct Mail.