Hillsdale College

How reducing friction for highly motivated traffic affects name conversion

Experiment ID: #21354

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 10/12/2015 - 10/21/2015

Hillsdale College was promoting a free first chapter of Dr. Larry Arnn’s book on Winston Churchill to grow their email list. Facebook advertising allowed them to target very specific audiences, which meant that our visitors presumably had a very high motivation to download the offer. While the page was converting more than 25% of visitors, we hypothesized that we could improve this conversion rate. Typically, motivation is the hardest conversion factor to affect. But since we were able to focus our targeting to highly motivated prospects, we wondered if the amount of copy on the page was too much, as it pushed the name capture form below the fold. We set up a treatment that removed the extra descriptive copy and reduced friction on the page.

Research Question

With highly motivated traffic, will a more concise call-to-action increase conversion rate?

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: 28.2%
T1: Short Copy 35.6% 26.2% 99.1%

This experiment has a required sample size of 296 in order to be valid. Since the experiment had a total sample size of 1,446, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 26.2% increase in conversion rate
× 0% increase in average gift

Key Learnings

This test shows that when you can hypertarget your traffic sources and drive highly motivated visitors, you might not need to “sell” as much. However, if we were attracting visitors from less targeted sources, we might want to re-test this control.


Experiment Documented by Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.

Question about experiment #4952

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.