Hillsdale College

How adding social proof on a donation page with a premium impacted donor conversion

Experiment ID: #53127

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 02/24/2021 - 03/22/2021

We wanted to test adding social proof across donation pages that have a DVD premium offer. Our hypothesis was this would increase the perceived value and credibility of the offer by showing the popularity of the course content.

Research Question

We believe that adding the number of course registrants for donation page visitors will achieve an increase in donor conversion.

Design

C: Control
T1: GASLOH DVD with Social Proof

Results

 Treatment NameRevenue per VisitorRelative DifferenceConfidenceAverage Gift
C: Control $6.15$107.42
T1: GASLOH DVD with Social Proof $7.9829.7% 99.4%$109.46

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 27.3% increase in conversion rate
× 1.9% increase in average gift
29.7% increase in revenue

Key Learnings

What we found across all traffic was that donations a 27.3% increase with a 99% level of confidence. Based on these results, we’re going to further test this concept by adding the number of DVDs that have been requested as well as test using survey quotes as an additional form of social proof. We also found two areas—tablet visitors and referral visitors—to further optimize that had a valid decrease in revenue on this treatment.

Observations of the impact on other visitor segments in the experiment:

  • Desktop visitors had a 29.5% increase in revenue with a 97% level of confidence.
  • Mobile visitors had a 68.3% increase in revenue with a 100% level of confidence.
  • Tablet visitors had a 90.6% decrease in revenue with a 100% level of confidence.
  • Returning visitors had a 53.7% increase in revenue with a 100% level of confidence.
  • Display visitors had a 51.2% increase in revenue with a 91% level of confidence.
  • Email visitors had a 36.9% increase in revenue with a 100% level of confidence.
  • Referral visitors had a 68.3% decrease in revenue with a 98% level of confidence.
  • Social visitors had a 909.1% increase in revenue with an 85% level of confidence.


Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #53127

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.