How focusing on the benefits of the offer impacted email acquisition
Buckner International
Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.
Experiment Summary
Timeframe: 07/20/2017 - 07/31/2017
Buckner International had been running a devotional offer for parents as a means of email acquisition for roughly 2 months. We had performed a series of experiments to boost the performance of the campaign but none had given us the kind of lift that we look for on a campaign like this.
We noticed that the cost per email had begun to rise over time. Instead of turning off the campaign, we decided to try out one more experiment. In this instance, we wanted to see if we could reframe the offer to highlight the benefits of prayer itself (not just the benefits of the offer) and incorporating actual scripture into the ad.
Research Question
Will focusing on the benefits of the action the offer promotes increase email acquisition for a biblically focused offer?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Scripture | 0.56% | ||
| T1: | With Scripture | 1.0% | 78.9% | 100.0% |
This experiment has a required sample size of 3,782 in order to be valid. Since the experiment had a total sample size of 81,994, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 78.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
This test teaches us the importance of testing each of the reasons/needs for someone to engage in an offer.
While the control’s benefit is perfectly fine, it is limited in value according to this audience. By connecting the offer to a deeper, more powerful value/desire in the audience (i.e. leaving no stone unturned for your kids versus making prayer more interesting/less boring), more people responded.
As a result, the new ad achieved a 79% increase in emails and reduced cost per email by 43%.