How creating a longer “process” impacted desktop donor conversion Experiment ID: #16509

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 5/23/2019 - 6/21/2019

In an effort to improve downstream donor conversion of people signing the statement of belief on the ADF blog, we decided to test out a new way to frame the process. The current control had the first step of signing the statement of belief and then the second step was a thank you page with a donation form on it. Our hypothesis was that we could improve donor conversion by making it feel more like a single process with multiple steps than two unique processes.

To do this, we added a “Step 1 of 2” on the statement of belief and then modified the donation page to make it feel like the second step rather than a complete new page.

Research Question

Will reframing the donor acquisition funnel to be a single process with multiple steps increase overall donor conversion?


C: Control
T1: Stepped Process


Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.71% $0.00
T1: Stepped Process 0.43% -39.5% 97.5% $0.00

This experiment has a required sample size of 5,551 in order to be valid. Since the experiment had a total sample size of 14,477, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 39.5% decrease in conversion rate
× 0% increase in average gift

Key Learnings

After running for a month, we saw a 40% decrease in donor conversion with the stepped process treatment. This was specifically with the desktop audience. The mobile audience did not see any statistically significant change in donor conversion but, with that audience, the donor conversion was much lower which may have accounted for no visible change. We also did not see any statistically significant change in the number of emails acquired.

Our hypothesis for this outcome is that the people signing the statement may have felt like the second step was a bait-and-switch. By making it feel like two separate processes, the solicitation did not feel like a requirement which increased the donors’ sense of generosity.

Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.