How offering a book as a premium with a donation vs at retail price impacts conversion rate and overall revenue
Time of Grace
Experiment Summary
Ended On: 02/16/2024
For Time of Grace’s monthly book promotion, we hypothesized that offering the book in exchange for a donation would result in increased transactions and revenue versus just offering it for purchase. Historically, Time of Grace has promoted their new books on their Shopify store at retail price. This test was accomplished by making a change to two areas of the funnel. The control vs treatment emails would have a change in the call-to-action copy. The control (for purchase) would lead them to Time of Grace’s Shopify page, whereas the treatment would lead them to a donation page that had the book as a premium on it.
Research Question
We believe that offering a new book as a premium gift in exchange for a donation rather than as a product for purchase for Time of Grace’s donor file will achieve an increase in transactions and revenue because users are more likely to be motivated by the generosity of the organization, and the value of receiving the gift for a donation.. because users are more likely to be motivated by the generosity of the organization, and the value of receiving the gift for a donation..
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.32% | $36.89 | ||
| T1: | Treatment #1 | 0.61% | 92.2% | 99.5% | $57.40 |
This experiment has a required sample size of 4,151 in order to be valid. Since the experiment had a total sample size of 17,057, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
92.2% increase in traffic
× 0% increase in conversion rate
× 55.6% increase in average gift
199.0% increase in revenue
Key Learnings
The key learning from this experiment is that offering a new book as a premium gift in exchange for a donation, instead of selling it at retail price, significantly increased the number of donations received. The treatment group saw a 92.2% increase in donations compared to the control group, with a high statistical level of confidence.
Not only did this result in an increase in donor/purchaser conversion rate, it also resulted in a valid 199% increase in revenue with a 98.9% level of confidence.
Control: 27 Purchases | $996 in Revenue | $36.89 Average Gift
Treatment 52 Donations |$2,985 in Revenue | $57.40 Average Gift
This approach also did not limit the ministry’s impact in actual copies “sold” with 53 units being purchased under the control and 55 units being requested as premiums with a donation.
This indicates that users are more motivated by the generosity of the organization and the perceived value of receiving a gift in exchange for their donation. Therefore, in future experimentation, it would be beneficial to continue offering premium gifts as incentives for donations.
This could be applied to other products or promotions as well, where offering a premium gift or exclusive content in exchange for a donation or purchase may motivate users to take action and increase overall transactions and revenue. The organization should consider implementing this strategy more frequently to leverage the increased motivation and generosity of their audience.