How adding a premium book offer affects donor revenue Experiment ID: #6145
Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.
Timeframe: 2/1/2017 - 2/13/2017
Hillsdale College was promoting their new online course, The U.S. Supreme Court. They were acquiring course subscribers and email addresses at a decent rate, but donor conversion was well below average, which led to a much higher cost-per-donor. They wanted to test whether adding a premium offer would increase donor conversion, by giving new donors something tangible to go along with the course material. They also hypothesized that their premium book, The U.S. Constitution: A Reader, would increase average gift since it was offered for any gift of $100 or more.
Since higher conversion rates and average gift compound to increase revenue, they thought this premium book offer could have a profound impact on revenue if successful. They launched an A/B test to find out.
How will adding a book as a premium offer affect conversion and revenue?
MECLABS Conversion Factors Targeted
C = 4m + 3v + 2( i - f) - 2a ©
Copyright 2015, MECLABS
|Treatment Name||Revenue per Visitor||Relative Difference||Confidence|
|T1:||Book Offer Premium||$1.42||53.8%||99.5%|
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 26.2% increase in conversion rate
× 21.9% increase in average gift
53.8% increase in revenue
The page with the premium offer increased revenue by 53.8%. This was due to the compounding increases in conversion rate (26.2%) and average gift (21.9%). This reminds us that the three factors that make up revenue (traffic, conversion rate, and average gift) all compound rather than add up. Simple addition would have led to a 48.1% increase. But the two happening in tandem led to a much more significant increase.
This is a great reminder of the power of launching tests that could potentially affect multiple FCORM (Flux Capacitor of Online Revenue Maximization) metrics.