How changing the value proposition to focus on “more content like this” affects donor conversion
Luther Seminary
Experiment Summary
Timeframe: 03/08/2023 - 03/26/2023
Luther Seminary has run this experiment across multiple of their content offers running in paid acquisition through Facebook including study guides and an online course for Revelation. They recently launched a new online course about the Book of Psalms, so we wanted to test this concept on the instant donation page to see if it would produce the same lift in conversion as it did for Revelation (834.6% lift with a 99.1% LoC).
The control donation page focused on reasons to give that were given were very high-level, wide-ranging, and broad. We hypothesized that if they focused the value proposition on just one element—particularly, the online courses that the user had just signed up for, they could improve donor conversion rate.
So, we drafted a new donation page with this “micro-value-proposition” and launched a test.
Research Question
We believe that focusing the value proposition on “more content like this” for online course enrollees will achieve an increase in conversion rate because specificity helps decrease cognitive friction.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 4.4% | $0.00 | ||
| T1: | Treatment | 9.1% | 105.2% | 98.3% | $0.00 |
This experiment has a required sample size of 225 in order to be valid. Since the experiment had a total sample size of 648, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 105.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running for more than three weeks, the single-value-proposition treatment produced a lift in conversion rate by 105.2% with a 98.2% level of confidence.
This experiment demonstrated that Luther Seminary might significantly increase its donation rate by focusing the value proposition on a single element — in this case, the online course it promotes. By providing a “micro-value-proposition,” which is more targeted and relevant, it helps decrease cognitive friction and increase conversions.
Furthermore, the experiment showed that the treatment was particularly effective for email visitors, desktop visitors, and returning visitors. Email visitors saw a 1564% increase in donations, desktop visitors had a 1350% increase in donations, and returning visitors had a massive 4817% increase in donations.
In future experimentation, organizations should focus on implementing a more targeted and relevant value proposition to improve conversion rates.