How a premium offer affects donor motivation
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 11/10/2016 - 11/14/2016
Dallas Theological Seminary had released a free online course on the book of Revelation. After visitors registered for the course, they were given the opportunity to give a gift to support the free courses. DTS had traditionally used commentary books as premiums for donations, but wanted to test the impact of removing the premium offer on donations and revenue.
They hypothesized that removing the book might result in more donations, though the average gift might be lower without the “anchor point” ($75, in this case) for the book offer.
They launched an A/B test to determine a winner.
Research Question
How does the presence of a premium offer affect donor conversion and revenue?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Original | 2.7% | $73.54 | ||
| T1: | no book | 4.1% | 55.1% | 99.8% | $48.15 |
This experiment has a required sample size of 1,167 in order to be valid. Since the experiment had a total sample size of 5,671, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 55.1% increase in conversion rate
× 34.5% decrease in average gift
1.6% increase in revenue
Key Learnings
The treatment with no premium book offer produced a 55.1% increase in donations. Without the “anchor point” of the book, average gift dropped by 34.5%, but there was no discernible impact on revenue.
This indicated something deeper about the motivation of their donors: without the premium offer, the value proposition was entirely centered around supporting the free online courses—and more donors responded. This suggests that DTS prospects are more motivated by an aspirational offer rather than a tactile offer.
The fact that the average gift for the control is right at $75 suggests that people either give to get the book, or don’t give at all. This may imply that people are conflating the offer with the incentive, and the aspirational aspect of the value proposition is clouded by the offer.
Because DTS can offer the premium throughout the course (in a banner) and directly through an email, they will move forward with the treatment. As a result of this experiment, DTS was able to grow their donor file significantly faster without affecting revenue.