How personal tone in a subject line affects open rate Experiment ID: #4116
Americans for Prosperity
Timeframe: 5/15/2016 - 5/20/2016
Americans for Prosperity had scheduled a send for an appeal offering membership to their exclusive Torchbearer club. A Torchbearer membership is a big ask — a monthly recurring donation — so we hypothesized that the email must be as high-touch as possible.
This experience for the reader began with the subject line. Since the sender was Tim Phillips, the CEO of AFP, we wanted the subject line to be very personal. The control subject line said “Bringing the greatest message in a generation”, which was relevant to the content, but didn’t seem like a subject line that he might actually write. So we developed a treatment that matched the radical redesign of the treatment email: “An invitation”.
We then tested the two emails in equal segments to determine a winner.
Will a more personal subject line result in a higher open rate?
|Treatment Name||Open Rate||Relative Difference||Confidence|
|T1:||Personal Subject Line||14.8%||44.7%||100.0%|
This experiment has a required sample size of 404 in order to be valid. Since the experiment had a total sample size of 14,613, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
44.7% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
The treatment subject line increased open rate by 44.7%. This indicates that creating a personal email experience begins with the subject line. If the reader doesn’t believe it’s coming from a person, they might be less likely to open the email. We also tested a radical redesign of the email and a completely new copy approach, which also produced a lift in clickthrough rate. This is a reminder of a key principle from the Fundraiser’s Creed: people give to people, not organizations or marketing machines.