How third-party credibility affects email acquisition rate Experiment ID: #6331

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 3/1/2017 - 3/20/2017

Hillsdale College had launched a course on the United States Supreme Court. Previous testing had already shown that showing a prospect the course schedule outperformed an image of two radio hosts that often promote the course. They had gathered some student testimonials that they wanted to test in place of the course schedule. They hypothesized that these testimonials might outperform the course schedule. Even though the course schedule gave more insight into what was on the other side, the testimonials gave third-party credibility to how good the content was on the other side.

They launched an A/B test to determine a winner.

Research Question

Will using student quotes as third-party credibility increase email acquisition rate?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: Course Schedule
T1: Student Quotes

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Course Schedule 25.1%
T1: Student Quotes 30.2% 20.5% 97.9%

This experiment has a required sample size of 582 in order to be valid. Since the experiment had a total sample size of 1,607, and the level of confidence is above 95%, the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 20.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

The student quotes produced a 20.5% lift in email acquisition over the course schedule. This shows that third-party validation from other people (even if they are relatively unknown) is a powerful motivator to accept an offer. It’s one thing to just see what’s on the other side, but it’s much more powerful to hear about the quality of the experience from other people.

This also may suggest that people are more motivated by quotes from other people than they are additional information. This principle is seen everywhere from Consumer Reports to Amazon reviews, but this experiment shows that it is accessible for nonprofit organizations to boost email acquisition rates.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.