How a radical redesign impacted donor conversion during a high urgency campaign
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 06/28/2021 - 08/13/2021
During a fiscal year end campaign, we were seeing lower donor conversion compared to the previous year so we decided to develop a treatment to the donation page for the remainder of the campaign to see if we could improve performance. We modified the headline from “Help DTS reach the world with solid biblical training” to “You can help DTS train students to reach the world with the gospel of Jesus Christ!” We also removed body copy that seemed organization-centric that referenced enrollment trends and their accredited degree programs and instead of featuring the high-caliber quality of the training gave examples of vocations that students pursue after their education and training (counseling, pastoring, etc.) to humanize the institution and presented this as an experiential outcome for how they would “share the good news and strengthen believers” which aligns with what the donor wants to accomplish through their gift. We also introduced copy that highlighted the option to give monthly.
Research Question
We believe that humanizing the donation page with experiential language for fiscal year end donors will achieve an increased donor conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 34.3% | $0.00 | ||
| T1: | FYE Treatment | 40.9% | 19.1% | 93.1% | $0.00 |
This experiment has a required sample size of 421 in order to be valid. Since the experiment had a total sample size of 723, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
Since the experiment only ran during the final 2 days of the campaign, we would need to run an additional test with similar language to truly test the concept without the existing urgency of the deadline that could be influencing results. The treatment donation page had a 19.1% increase in conversion rate with 93% level of confidence. We want to take this directional lift in gifts on the page and see where else we could apply this learning.
While we did not reach statistical significance to validate the experiment, we did observe some notable changes to other metrics due to the experiment that are outlined below:
- Revenue experienced a 47.3% decrease with a 97% level of confidence.
Observations of the impact on other visitor segments in the experiment:
- New visitors had a 39.8% increase in donations with a 99% level of confidence.
- Email visitors had a 23.9% increase in donations with a 98% level of confidence.
- Tablet visitors had a 100.0% decrease in donations with an 87% level of confidence.
- 55-64 visitors had an 88.5% increase in donations with a 93% level of confidence.
- Female visitors had a 70.9% increase in donations with a 96% level of confidence.
- 25-34 visitors had a 76.4% decrease in donations with a 96% level of confidence.