How the messaging and sender in a year-end email impacts donor conversion Experiment ID: #19472
Timeframe: 12/19/2019 - 12/19/2019
During EWTN’s year-end campaign, they wanted to test the impact on a different sender for one of their appeals. The appeals typically come from the CEO of the organization, Michael Warsaw. The control email came from Michael with messaging around what EWTN wanted/needed to accomplish in 2020. He communicated specific goals and initiatives in the copy and had a more serious tone to it. We tested this approach against an email from the VP of Mission Advancement, Scott Brown. This treatment email also highlighted goals for 2020 in it as well, but took a more conversational/lighter tone to it.
Would a different tone and sender increase donor conversion?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|C:||Michael P. Warsaw as sender||0.06%|
|T1:||Scott F. Brown as sender||0.06%||-8.1%||35.7%|
This experiment has a required sample size of 1,746,352 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
We learned that the tone and sender of this email had no impact on donor conversion. While the treatment proved to not increase donor conversion, it also showed us that a different sender – other than the organization’s CEO – could be the sender of their email appeals and have no NEGATIVE impact on conversion. This is a great learning and something we will test again outside of a high-urgency campaign.