How offering up a digital version of Angels and Demons increased email acquisition rates for Pathway to Victory
First Baptist Dallas
Experiment Summary
Timeframe: 04/14/2022 - 05/11/2022
Pathway to Victory is a ministry who has built a fundraising program around the offer of premiums—books, pamphlets, brochures, etc. Most of their new name acquisition efforts (not donor acquisition) focused around offering up brochures and pamphlets that were then mailed to the recipient. Because this fulfillment process usually took a month, we hypothesized that offering an instant digital download of the same exact premium, in PDF format, would drive a higher email acquisition rate at a much lower cost to acquire.
This is an extension of an experiment we tested with a single premium in April; for this experiment, we were testing the Angels and Demons resource.
Research Question
We believe that offering a digital download for a Facebook audience will achieve a better email conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Physical Premium | 0.22% | ||
| T1: | Digital Premium | 1.5% | 558.1% | 100.0% |
This experiment has a required sample size of 302 in order to be valid. Since the experiment had a total sample size of 303,555, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 558.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
Pathway to Victory was able to achieve a massive 558% increase in email conversion rates, with a 100% level of confidence. Moreover, the cost to acquire an email address was 67% lower on the digital premium.
We believe this positive result is due to a combination of factors:
- By offering up immediate gratification through a digital download, versus potentially waiting up to a month to receive your resource in the mail, the audience was much more likely to sign up to receive it.
- By not requiring a full mailing address for the digital version, we removed some friction in the sign-up process.
We will continue to test this hypothesis, as well as track the downstream impact on revenue of these email sign-ups.