How reducing the giving experience from 3 steps to 2 steps impacts donor conversion rate (pt. 2)
The Navigators
Experiment Summary
Timeframe: 07/20/2022 - 08/06/2022
Since transitioning to a new giving platform, The Navigators’ giving process has been split up into multiple steps and pages. When we run an ebook offer on Facebook, people now have the following experience: they fill out the form to get the ebook, get a confirmation page with the donation value prop language, click through to donate, go to a generic donation page to select their gift amount, and then a checkout process. Our hypothesis is that all these steps are causing friction for the potential donor. As much as possible, we wanted to consolidate the steps by running a redirect experiment on one of their ebook offers.
We had already run this experiment with a previous offer but we did not get a large enough sample size to validate the results. So, we re-ran this experiment with their most popular offer in order to achieve statistical validity in our results.
The treatment for this experiment reduced the process down to 2 steps so that the value proposition language was on the same page as the gift array. We also made this new page our ideal donation page by creating a tabbed form and moving the gift array beneath the copy. This is to mimic what the donation pages used to look like for this organization.
Research Question
We believe that reducing the giving experiences from 3 steps to steps for prospective donors from an ebook offer will achieve a higher donor conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control - 3 step process | 0.71% | |||
| T1: | Treatment - 2 two step process | 1.4% | 97.2% | 99.9% |
This experiment has a required sample size of 1,690 in order to be valid. Since the experiment had a total sample size of 10,182, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 97.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
The results for this experiment produced a valid 97% lift in donor conversion rate, with a level of confidence of 99%. These results indicate that the new giving experience is indeed causing friction for prospective donors. Reducing the steps down to 2 steps made it easier for the user to make their donation. In addition to that, the adjusted donation page format introduced other variables that may have also contributed to the significant lift in donations. Ideally, we would like to test adding the actual payment information and process on the same page so that we can further consolidate the steps and reduce the percentage of those who abandon the process.