How does following up a direct mail renewal note with an email impact conversions
First Baptist Dallas
Experiment Summary
Ended On: 03/04/2024
Pathway to Victory is the broadcast ministry of First Dallas.
6 times a year, PTV puts out Pathway Magazine. A resource that includes daily devotionals for 2 months, articles by Dr. Jeffress and opportunities to learn about PTV’s newest books.
An individual can sign up to receive 3 free issues of Pathway Magazine, but on the 3rd issue, PTV attaches a message that this is the last issue unless a missional gift of $20 is made to support PTV.
The effectiveness of this note has been minimal, therefore we decided to test to see if a follow-up email would generate additional renewals.
Research Question
We believe that by sending a follow up email for Pathway Magazine subscribers that are ready for renewal will achieve a higher conversion rate because this is an additional touch point to the direct mail note.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.45% | ||
| T1: | Treatment #1 | 1.0% | 132.1% | 99.2% |
This experiment has a required sample size of 1,591 in order to be valid. Since the experiment had a total sample size of 6,000, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 132.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this A/B experiment conducted by Pathway to Victory is that sending a follow-up email to Pathway Magazine subscribers who are due for renewal significantly increased donations compared to the traditional direct mail note. The treatment group, which received the follow-up email, experienced a 132.1% increase in donations with a 99.2% statistical level of confidence.
This experiment demonstrates the importance of utilizing multiple touchpoints to engage with subscribers and donors. By incorporating email communication as a follow-up to direct mail efforts, organizations like Pathway to Victory can enhance donor retention and increase overall engagement.
Key points of this experiment.
- Only contacts with email addresses were included in the experiment
- Donations are all donations given either through direct mail or email
- Treatment saw 17 gifts through direct mail, 12 gifts through email, and 1 donor contributed via email and direct mail for this appeal