How a more modern donation form user experience affects donor conversion | NextAfter
Hillsdale College

How a more modern donation form user experience affects donor conversion

Experiment ID: #3552

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 03/16/2016 - 03/23/2016

The donation form for Hillsdale has gone through a series of iterations to land at its current state. The form is not highly styled but it is laid out in such a way that we felt it would be easiest for donors to fill out. With the recent shifts taking place online with regards to user experience, we realized that the form was not as modern looking as many of the other options out there.

We decided to try improving the user experience by giving the form some more modern touches. This included minimizing the visual size of the form by removing the labels from above the fields. Then, so that users would know what information was required for each field, we implemented a technique known as adaptive placeholders. Adaptive placeholders sit in the field until a user focuses on the field, at which point the placeholder floats up above the field. You can see this form in action here.

We then tested this new form against the our original control.

Research Question

Will a form that we believe produces a better user experience positively impact donor conversion?

Design

C: Old Form
T1: Pretty Form

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Old Form 3.0%
T1: Pretty Form 0.91% -69.7% 99.9%

This experiment has a required sample size of 335 in order to be valid. Since the experiment had a total sample size of 1,956, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 69.7% decrease in conversion rate
× 21.1% increase in average gift
63.3% decrease in revenue

Key Learnings

This experiment result was heartbreaking. We were so certain that the more modern form would create a much better user experience which would lead to more donations. At a minimum, we expected it to have no significant impact. We never thought it would so dramatically reduce the conversion rate of the page!

Our hypothesis for this result is that the new user experience was a distraction for donor. It was far different than they were use to which made it more work to give a gift. These two factors ultimately led to abandonment of the donation process which reduced conversion rate by 69.7%


Experiment Documented by NextAfter

Question about experiment #3552

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.