How adding stronger value proposition to the primary donation page impacts donor conversion
Boys Town
Experiment Summary
Timeframe: 01/18/2019 - 02/04/2019
Boys Town had done a series of tests on their primary donation page around value proposition. Most of the experiments tested the control that had a single image at the top of it against removing the image and communicating a longer value proposition through paragraphs of copy. In all of these tests, they found that the single image at the top of their page with the message, “Thank you, you’re not just saving one child” consistently performed best. Taking these learnings, we hypothesized that we might be able to increase donor acquisition if we tested the value proposition within the image itself. The treatment version clearly communicated a case for support and the impact a donor would have with their gift.
Research Question
Would a stronger value proposition increase donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 53.5% | $0.00 | ||
| T1: | Value prop banner copy | 44.1% | -17.6% | 99.7% | $0.00 |
This experiment has a required sample size of 216 in order to be valid. Since the experiment had a total sample size of 986, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 17.6% decrease in conversion rate
× 0% increase in average gift
Key Learnings
The treatment decreased overall donor conversion. Since this experiment ran on the primary donation page where a variety of visitors come with varying motivations, we dug into the audience break down a bit to try and get a better understanding of the results. What is interesting is that not all audience segments had a decrease in conversion. In fact, when we looked at new visitors on mobile devices we found that we could increase donor conversion by 12%.
Moving forward, there are two approaches we propose testing. First, there is a hypothesis that the majority of people giving on this page are from Direct Mail. Having a specific value proposition that is tied to Direct Mail may reinforce that hypothesis and result in higher donor conversion. Second, there has been significant testing around value proposition on this page, all resulting in not being able to lift conversion. We propose testing another element that could impact overall revenue from the page like testing the suggested giving amounts.