Using “urgent and important” in campaign email subject line
Hillsdale College
Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.
Experiment Summary
Timeframe: 09/30/2020 - 10/05/2020
For the final email of the Constitution Day 2020 HFC, we used “tick tick tick” for passage of time, but tested “urgent and important” to gain more engagement as people were potentially fatigued with the campaign.
Research Question
Is it worth deploying “urgent and important” in a subject line?
Design
Results
| Treatment Name | Open Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Tick...tick...tick... | 28.5% | ||
| T1: | Some important and urgent news | 29.4% | 3.1% | 100.0% |
This experiment has a required sample size of 20,288 in order to be valid. Since the experiment had a total sample size of 504,473, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
3.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
People were more likely to open “urgent and important news” (3% lift) but were less likely to engage (-6% lift on click to open) so we disappointed our recipients and failed to meet their expectation of urgent and important with this email.