How adding a premium book offer affects donor revenue Experiment ID: #6145

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 2/1/2017 - 2/13/2017

Hillsdale College was promoting their new online course, The U.S. Supreme Court. They were acquiring course subscribers and email addresses at a decent rate, but donor conversion was well below average, which led to a much higher cost-per-donor. They wanted to test whether adding a premium offer would increase donor conversion, by giving new donors something tangible to go along with the course material. They also hypothesized that their premium book, The U.S. Constitution: A Reader, would increase average gift since it was offered for any gift of $100 or more.

Since higher conversion rates and average gift compound to increase revenue, they thought this premium book offer could have a profound impact on revenue if successful. They launched an A/B test to find out.

Research Question

How will adding a book as a premium offer affect conversion and revenue?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: No Premium
T1: Book Offer Premium

Results

Treatment Name Revenue per Visitor Relative Difference Confidence
C: No Premium $0.92
T1: Book Offer Premium $1.42 53.8% 99.5%

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 26.2% increase in conversion rate
× 21.9% increase in average gift
53.8% increase in revenue

Key Learnings

The page with the premium offer increased revenue by 53.8%. This was due to the compounding increases in conversion rate (26.2%) and average gift (21.9%). This reminds us that the three factors that make up revenue (traffic, conversion rate, and average gift) all compound rather than add up. Simple addition would have led to a 48.1% increase. But the two happening in tandem led to a much more significant increase.

This is a great reminder of the power of launching tests that could potentially affect multiple FCORM (Flux Capacitor of Online Revenue Maximization) metrics.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.





Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.


Pick your format of choice:


Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.