How a more modern donation form user experience affects donor conversion
Hillsdale College
Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.
Experiment Summary
Timeframe: 03/16/2016 - 03/23/2016
The donation form for Hillsdale has gone through a series of iterations to land at its current state. The form is not highly styled but it is laid out in such a way that we felt it would be easiest for donors to fill out. With the recent shifts taking place online with regards to user experience, we realized that the form was not as modern looking as many of the other options out there.
We decided to try improving the user experience by giving the form some more modern touches. This included minimizing the visual size of the form by removing the labels from above the fields. Then, so that users would know what information was required for each field, we implemented a technique known as adaptive placeholders. Adaptive placeholders sit in the field until a user focuses on the field, at which point the placeholder floats up above the field. You can see this form in action here.
We then tested this new form against the our original control.
Research Question
Will a form that we believe produces a better user experience positively impact donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Old Form | 3.0% | ||
| T1: | Pretty Form | 0.91% | -69.7% | 99.9% |
This experiment has a required sample size of 335 in order to be valid. Since the experiment had a total sample size of 1,956, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 69.7% decrease in conversion rate
× 21.1% increase in average gift
63.3% decrease in revenue
Key Learnings
This experiment result was heartbreaking. We were so certain that the more modern form would create a much better user experience which would lead to more donations. At a minimum, we expected it to have no significant impact. We never thought it would so dramatically reduce the conversion rate of the page!
Our hypothesis for this result is that the new user experience was a distraction for donor. It was far different than they were use to which made it more work to give a gift. These two factors ultimately led to abandonment of the donation process which reduced conversion rate by 69.7%