How moving the donation page copy to “below the fold” impacted donor conversion rate
Leadership Institute
Experiment Summary
Timeframe: 05/13/2022 - 05/23/2022
As a part of a recent acquisition campaign in the “text to give” channel, the MMS partner noted that the donation page copy was “too long” and felt like it should be moved to “below the fold” (a nice newspaper term), or they wouldn’t agree to promote the offer because it was likely to not be as productive as what their “best practices” would dictate.
So, to appease them, the Leadership Institute agreed to run an A/B experiment to track the performance on the campaign and see how it impacted overall giving from their audiences.
Research Question
We believe that moving the donation page copy to “below the fold” to streamline the giving experience for prospective donors in the text to give channel will achieve decrease revenue.
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | $7.62 | $52.12 | ||
| T1: | Shortened Treatment | $3.23 | -57.6% | 91.4% | $22.08 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.
Key Learnings
With a 91.3% level of confidence, we observed a decrease in revenue of -57.6%, largely driven by a decrease in the average gift of more than 50% coming from visitors flowing through the treatment variant.
What’s interesting is that there was the exactly same number of transactions between the treatment and the control experiences (0% difference), however, the generosity of the donors in the control experience was nearly twice as much as the treatment visitors.
The question is: Why?
In reviewing the transactions, we noticed that there were not any outlier gifts in the overall experiment, but there were 4 gifts at or above $100 that came through the control experience.
When you consider the channel used to drive visitors to this page (MMS) — it’s clear what is behind this increase in generosity. The MMS channel messaging is designed to get a click, but after a visitor arrives on the donation page — the lack of value proposition and reason to give communicated on the page was enough to drive visitors in the treatment to give at lower amounts. Alternatively, when the value proposition to give in the control experience was placed where it normally has been, it showed that donors to the control decided not only to give at the same rate as those in the treatment experience, but they were more informed on WHY they should give and WHO they were giving to — thereby increasing the amount they decided to give.
When possible, it’s highly recommended — especially in campaigns that are not driving donor-motivated clicks (like from your house email file) — to include copy that properly articulates the value proposition on the donation page, so that once they arrive on the page, they’ll not only decide to give, but when they do — they’ll give more generously.