Hillsdale College

How clarifying the value proposition affects email acquisition rate

Experiment ID: #2684

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 11/14/2015 - 11/24/2015

Hillsdale College was promoting their popular Constitution 101 course with a page that had been through a series of tests centered around copy and the email acquisition form. Through all of these, the top banner had remained the same. They hypothesized that the separation of the headline from the rest of the copy might be creating a “break” between the two sections. They designed a treatment that removed the “banner” section and closely joined it to the body copy, in an effort to reduce friction and increase the clarity of the value proposition. Also, they tested an alternate headline that tested the value proposition offering “Instant Free Access” to the course, hoping both of these changes would result in an increase in emails acquired.

Research Question

Can we affect email acquisition rate by positioning the headline in greater proximity to the body copy and increasing the clarity of the value proposition?


C: Control - Banner & Know the Constitution
T1: Instant Free Access - No Banner
T2: Know the constitution - No Banner


  Treatment Name Conv. Rate Relative Difference Confidence
C: Control - Banner & Know the Constitution 37.5%
T1: Instant Free Access - No Banner 40.0% 6.6% 92.5%
T2: Know the constitution - No Banner 41.0% 9.2% 98.7%

This experiment has a required sample size of 2,327 in order to be valid. Since the experiment had a total sample size of 7,473, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 9.2% increase in conversion rate
× 0% increase in average gift

Key Learnings

Hillsdale saw a 9.2% lift in email acquisition from these simple design changes, showing that proximity matters to visitors. If we separate the attention-grabbing headline too far from the body copy, it reduces the likelihood that they visitors will read both. Additionally, there were friction factors at work as well — reducing the overall height of the page brought the signup form closer to the top of the page, which led to reduced scrolling without affecting the amount of value proposition copy on the page.

Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #2684

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.