How switching to a digital premium for Parables of Jesus impacts email acquisition rates for Pathway to Victory
First Baptist Dallas
Experiment Summary
Timeframe: 04/14/2022 - 05/11/2022
Pathway to Victory is a ministry who has built a fundraising program around the offer of premiums—books, pamphlets, brochures, etc. Most of their new name acquisition efforts (not donor acquisition) focused around offering up brochures and pamphlets that were then mailed to the recipient. Because this fulfillment process usually took a month, we hypothesized that offering an instant digital download of the same exact premium, in PDF format, would drive a higher email acquisition rate at a much lower cost to acquire.
This is an extension of an experiment we tested with a single premium in April; for this experiment, we were testing the Parables of Jesus
Research Question
We believe that offering a digital premium for a Facebook audience will achieve a change in email acquisition rates.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.08% | ||
| T1: | Treatment #1 | 0.16% | 111.5% | 100.0% |
This experiment has a required sample size of 11,106 in order to be valid. Since the experiment had a total sample size of 689,941, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 111.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
Pathway to Victory was able to achieve a 115% increase in email conversion rates, with a 100% level of confidence. Moreover, the cost to acquire an email address was 6% lower on the digital premium.
We believe this positive result is due to a combination of factors:
- By offering up immediate gratification through a digital download, versus potentially waiting up to a month to receive your resource in the mail, the audience was much more likely to sign up to receive it.
- By not requiring a full mailing address for the digital version, we removed some friction in the sign-up process.
We will continue to test this hypothesis, as well as track the downstream impact on revenue of these email sign-ups.