How reinforcing the value of a premium offer impacts donor conversion
FamilyLife
FamilyLife® has been committed to helping individuals find biblical help for their marriage and family relationships. Through the Weekend to Remember® marriage getaways, FamilyLife Today® radio broadcasts, The Art of Marriage® video event, and the many other resources and content, God has used FamilyLife to restore hope for millions of couples and transform their lives.
Experiment Summary
Timeframe: 05/16/2022 - 06/13/2022
FamilyLife is a Christian ministry serving families worldwide. FamilyLife uses multiple premium offers on social media to acquire new donors. Their most popular premium is the Seeing Beautiful Again Devotional. They wanted to see if reinforcing the value proposition of the premium would increase donor conversion by making it more appealing to donors. They did this by creating a treatment that included social proof of what other donors said about the resource as well as changing the headline of the page to be more value driven. Instead of, “Seeing Beautiful Again” the treatment headline read, ” Gain healthier ways to process your pain through truth-based perspectives,” which we believed better emphasized the value the resource would bring to donors. We split traffic 50/50 between both pages and ran the test for 25 days.
Research Question
We believe that reinforcing the value of a premium offer on the donation page will increase donor conversion.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 4.7% | $0.00 | ||
| T1: | Reinforcing Value Preposition of the Book | 5.2% | 9.7% | 34.7% | $0.00 |
This experiment has a required sample size of 17,328 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The changes made to reinforce the value of the premium did not have a statistically significant impact on donor conversion or donor revenue. While we saw user segments having conflicting impact, higher donor conversion on new visitors vs returning visitors, no segment’s impact was able to be validated. Instead of focusing on the value proposition of the premium, we will shift our strategy to testing tactical elements on the page to see if that will improve conversion.