Discovering the ideal offer to fit the motivation of the visitors
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Timeframe: 12/20/2017 - 01/01/2018
Through a series of previous experiments, we had found that an in-article advertisement for a related assessment moved to the bottom of the article, would yield a significant boost in advertising clicks. As part of an iterative testing plan, we next wanted to discover what type of offer would be most appealing to the audience.
We decided to test the assessment-offer against a product-offer that they could receive for a gift of any amount. Now, we knew that we would receive less conversions but a donor conversion would be more valuable than an email conversion. The question was, what is the trade-off between the two offers; i.e. how many donors could we acquire vs. how many email addresses could we acquire?
Research Question
Which offer (assessment or product) is most appealing to the article readers?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Assessment Offer | 2.9% | ||
| T1: | Premium Donation Offer | 0.05% | -98.3% | 100.0% |
This experiment has a required sample size of 138 in order to be valid. Since the experiment had a total sample size of 169,881, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 98.3% decrease in conversion rate
× 0% increase in average gift
Key Learnings
Based upon this experiment, we found that we could acquire over 5,700% more email addresses than donors. This was also run during the last weeks of December when visitors are more motivated to give. This would indicate that the audience reading the articles is not a donor-motivated audience. This is a critical finding that will allow us to focus the conversion efforts further down the funnel once the email has been acquired.
* As a note, the table above reflects emails acquired vs. donors acquired. It is not a straight apples to apples comparison of “emails acquired.”