How creating a longer “process” impacted desktop donor conversion
Alliance Defending Freedom
Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.
Experiment Summary
Timeframe: 05/23/2019 - 06/21/2019
In an effort to improve downstream donor conversion of people signing the statement of belief on the ADF blog, we decided to test out a new way to frame the process. The current control had the first step of signing the statement of belief and then the second step was a thank you page with a donation form on it. Our hypothesis was that we could improve donor conversion by making it feel more like a single process with multiple steps than two unique processes.
To do this, we added a “Step 1 of 2” on the statement of belief and then modified the donation page to make it feel like the second step rather than a complete new page.
Research Question
Will reframing the donor acquisition funnel to be a single process with multiple steps increase overall donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.71% | $0.00 | ||
| T1: | Stepped Process | 0.43% | -39.5% | 97.5% |
This experiment has a required sample size of 5,551 in order to be valid. Since the experiment had a total sample size of 14,477, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 39.5% decrease in conversion rate
× 0% increase in average gift
Key Learnings
After running for a month, we saw a 40% decrease in donor conversion with the stepped process treatment. This was specifically with the desktop audience. The mobile audience did not see any statistically significant change in donor conversion but, with that audience, the donor conversion was much lower which may have accounted for no visible change. We also did not see any statistically significant change in the number of emails acquired.
Our hypothesis for this outcome is that the people signing the statement may have felt like the second step was a bait-and-switch. By making it feel like two separate processes, the solicitation did not feel like a requirement which increased the donors’ sense of generosity.