How third-party credibility affects email acquisition rate Experiment ID: #6331
Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.
Timeframe: 3/1/2017 - 3/20/2017
Hillsdale College had launched a course on the United States Supreme Court. Previous testing had already shown that showing a prospect the course schedule outperformed an image of two radio hosts that often promote the course. They had gathered some student testimonials that they wanted to test in place of the course schedule. They hypothesized that these testimonials might outperform the course schedule. Even though the course schedule gave more insight into what was on the other side, the testimonials gave third-party credibility to how good the content was on the other side.
They launched an A/B test to determine a winner.
Will using student quotes as third-party credibility increase email acquisition rate?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 582 in order to be valid. Since the experiment had a total sample size of 1,607, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 20.5% increase in conversion rate
× 0% increase in average gift
The student quotes produced a 20.5% lift in email acquisition over the course schedule. This shows that third-party validation from other people (even if they are relatively unknown) is a powerful motivator to accept an offer. It’s one thing to just see what’s on the other side, but it’s much more powerful to hear about the quality of the experience from other people.
This also may suggest that people are more motivated by quotes from other people than they are additional information. This principle is seen everywhere from Consumer Reports to Amazon reviews, but this experiment shows that it is accessible for nonprofit organizations to boost email acquisition rates.