How a radical donation page treatment impacts acquisition revenue
Buckner International
Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.
Experiment Summary
Timeframe: 07/01/2019 - 08/03/2019
After running for nearly six months, we had not been able to achieve a statistically significant lift to the donation page used with Buckner’s 7 Scripture offer. After multiple iterative approaches, we decided to test a radical redesign of the page. We combined multiple factors based upon positive results seen with similar organizations.
The down side to this approach is that it makes it difficult to isolate exactly what helped move the needle if a lift is achieved. However, the overall goal was to lift revenue and if that could be achieved then knowing the exact reason for the lift became less important.
Research Question
Will a radical redesign of the donation page increase overall revenue to the acquisition campaign?
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | $1.09 | $24.70 | ||
| T1: | Courtney's Version | $1.23 | 12.8% | 74.3% | $28.52 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.
Key Learnings
While we did not reach a statistical level of significance, we did see an increased average gift from the treatment version. Initial results saw the treatment also have a higher conversion rate but this balanced out after a few weeks of running. This is a good reminder of why it is so critical to ensure there is both a significant amount of time and total conversions before making a call on any experiment.
Our current hypothesis for the increase in average gift is that the treatment makes a targeted ask of $50 while the control version asks for $25. However, given all of the other factors that changed, we will need to isolate this factor to prove it out in the future.