How reducing navigation affects donor conversion
Americans for Prosperity
Experiment Summary
Timeframe: 08/02/2016 - 09/08/2016
Americans for Prosperity Foundation was preparing to launch a website redesign. In the few weeks they had before launch, wanted to run a quick donation page experiment to see if they could lift conversion.
The current donation page had a large navigation menu at the top that gave the visitor plenty of opportunities to leave the page before making a gift. They knew that removing these would prevent “unsupervised thinking” and they hypothesized that it would increase conversion.
They used Adobe Target to create a treatment with this navigation removed, and launched an A/B test to measure the winner.
Research Question
Will removing donation page navigation increase donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | With Navigation | 0.70% | ||
| T1: | Without Navigation | 2.1% | 195.1% | 86.4% |
This experiment has a required sample size of 583 in order to be valid. Since the experiment had a total sample size of 623, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
The treatment with no navigation produced a 195.1% increase in conversion, although the results came just short of statistical validity. They ended the test before arriving at validity to launch the new site, although the test will be repeated with the new design.