First Baptist Dallas

How switching to a digital premium for Parables of Jesus impacts email acquisition rates for Pathway to Victory

Experiment ID: #93842

First Baptist Dallas

Experiment Summary

Timeframe: 04/14/2022 - 05/11/2022

Pathway to Victory is a ministry who has built a fundraising program around the offer of premiums—books, pamphlets, brochures, etc. Most of their new name acquisition efforts (not donor acquisition) focused around offering up brochures and pamphlets that were then mailed to the recipient. Because this fulfillment process usually took a month, we hypothesized that offering an instant digital download of the same exact premium, in PDF format, would drive a higher email acquisition rate at a much lower cost to acquire.

This is an extension of an experiment we tested with a single premium in April; for this experiment, we were testing the Parables of Jesus

Research Question

We believe that offering a digital premium for a Facebook audience will achieve a change in email acquisition rates.


C: Control
T1: Treatment #1


  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.08%
T1: Treatment #1 0.16% 111.5% 100.0%

This experiment has a required sample size of 11,106 in order to be valid. Since the experiment had a total sample size of 689,941, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 111.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

Pathway to Victory was able to achieve a 115% increase in email conversion rates, with a 100% level of confidence. Moreover, the cost to acquire an email address was 6% lower on the digital premium.

We believe this positive result is due to a combination of factors:

  1. By offering up immediate gratification through a digital download, versus potentially waiting up to a month to receive your resource in the mail, the audience was much more likely to sign up to receive it.
  2. By not requiring a full mailing address for the digital version, we removed some friction in the sign-up process.

We will continue to test this hypothesis, as well as track the downstream impact on revenue of these email sign-ups.

Experiment Documented by NextAfter

Question about experiment #93842

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.