First Baptist Dallas

How does following up a direct mail renewal note with an email impact conversions

Experiment ID: #171182

First Baptist Dallas

Experiment Summary

Ended On: 03/04/2024

Pathway to Victory is the broadcast ministry of First Dallas.

6 times a year, PTV puts out Pathway Magazine. A resource that includes daily devotionals for 2 months, articles by Dr. Jeffress and opportunities to learn about PTV’s newest books.

An individual can sign up to receive 3 free issues of Pathway Magazine, but on the 3rd issue, PTV attaches a message that this is the last issue unless a missional gift of $20 is made to support PTV.

The effectiveness of this note has been minimal, therefore we decided to test to see if a follow-up email would generate additional renewals.

Research Question

We believe that by sending a follow up email for Pathway Magazine subscribers that are ready for renewal will achieve a higher conversion rate because this is an additional touch point to the direct mail note.


C: Control
T1: Treatment #1


 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.45%
T1: Treatment #1 1.0%132.1% 99.2%

This experiment has a required sample size of 1,591 in order to be valid. Since the experiment had a total sample size of 6,000, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 132.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

The key learning from this A/B experiment conducted by Pathway to Victory is that sending a follow-up email to Pathway Magazine subscribers who are due for renewal significantly increased donations compared to the traditional direct mail note. The treatment group, which received the follow-up email, experienced a 132.1% increase in donations with a 99.2% statistical level of confidence.

This experiment demonstrates the importance of utilizing multiple touchpoints to engage with subscribers and donors. By incorporating email communication as a follow-up to direct mail efforts, organizations like Pathway to Victory can enhance donor retention and increase overall engagement.

Key points of this experiment.

  • Only contacts with email addresses were included in the experiment
  • Donations are all donations given either through direct mail or email
  • Treatment saw 17 gifts through direct mail, 12 gifts through email, and 1 donor contributed via email and direct mail for this appeal

Experiment Documented by NextAfter

Question about experiment #171182

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.