Children’s Lit – HFC Email 1 Subject Line
Hillsdale College
Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.
Experiment Summary
Timeframe: 05/01/2020 - 05/04/2020
Tested vague whimsical saying related to our value prop vs. a vague subject line that intimated an ask.
Research Question
Does a seemingly random subject line outperform a an ask hint subject line?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Once upon a time | 23.8% | ||
| T1: | You're invited to make it a success | 23.0% | -3.3% | 100.0% |
This experiment has a required sample size of 22,080 in order to be valid. Since the experiment had a total sample size of 1,060,194, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
3.3% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
“Once upon a time” outperformed on opens AND click-to-opens, indicating the hint of fundraising intent for “you’re invited to make it a success” reduced engagement.