How targeting direct mail prospects with digital advertising affected direct mail revenue Experiment ID: #7533

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 8/1/2017 - 10/2/2017

Hillsdale College had significant investments in their direct mail program and wanted to see if they could optimize the return by investing in digital marketing that would target these prospects in the time period that they received a direct mail letter.

They created a control audience that was excluded from seeing any ads, and a treatment audience that would see a rotation of brand and course offering ads (but no donation-centric ads) for a two week period before the mail piece hit mailboxes and for two weeks after, the prime time in which a donor would respond. They optimized the ads for reach—aiming to show ads to as large a percentage of the target group as possible. The goal of this test was to lift direct mail revenue—not to add an additional segment of online revenue. 

They spent just shy of $1,000 to show ads to the treatment audience and waited to see the results come in.

Research Question

Will showing ads to a targeted group of direct mail recipients result in increased revenue from the treatment group?


C: Direct mail letter only
T1: Direct mail plus brand ads


Treatment Name Revenue per Visitor Relative Difference Confidence Average Gift
C: Direct mail letter only $0.46 $61.77
T1: Direct mail plus brand ads $1.18 154.5% 100.0% $46.32

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 239.4% increase in conversion rate
× 25.0% decrease in average gift
154.5% increase in revenue

Key Learnings

Just increasing the number of brand impressions during that time period to a targeted audience resulted in a 145.5% increase in revenue from the treatment group. This showed an incredible value in multichannel targeting—even if the digital ads weren’t focused on direct response.

While this test is quite promising (especially for an August appeal), more testing must be done to ensure that it holds through all seasons and a variety of direct mail offers.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.