How visually increasing the security of a page impacts conversion - NextAfter
Hillsdale College

How visually increasing the security of a page impacts conversion

Experiment ID: #5755

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 11/22/2016 - 12/07/2016

Hillsdale College has been iteratively testing the instant donor conversion page that follow each of the various email acquisition offers. Over time, we have found a unique combination of value proposition, page layout, and donation form design that has produced an incremental donor conversion. However, we’re always looking to find more ways to improve it.

One aspect that we have not been able to significantly improve is the anxiety associated with making a gift. Just adding security seals did not have much impact on conversion. However, we had the hypothesis that if we focused the visual reinforcement of the security on just the most sensitive data, it may be more impactful.

Research Question

Does visually reinforcing the security of the credit card section improve conversion?

Design

C: Control
T1: Credit Card Security Section

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 7.8%
T1: Credit Card Security Section 8.6% 9.5% 95.7%

This experiment has a required sample size of 10,396 in order to be valid. Since the experiment had a total sample size of 22,159, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 9.5% increase in conversion rate
× 0.2% increase in average gift
9.8% increase in revenue

Key Learnings

When we were able to visually reinforce the security of the credit card section by adding a gray box around it to visually separate it from the rest of the form and then included a simple security symbol. These small changes resulted in a 9.5% increase to donation page conversion.


Experiment Documented by NextAfter

Question about experiment #5755

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.