How campaign first messaging impacts clicks from donors in Giving Tuesday Now emails Experiment ID: #21240
Timeframe: 5/5/2020 - 5/5/2020
On Giving Tuesday Now, EWTN ran a campaign focused around acquiring recurring donors. Because Giving Tuesday Now is new to people, we were curious if that message would resonate with donors and nondonors on EWTN’s email file. We tested this by using mission based messaging at the beginning of the email and then later talked about Giving Tuesday Now for the control, and developed a treatment used Giving Tuesday Now focused messaging at the beginning of the email and then later used missional messaging. We tested this with two different segments – donors and nondonors to see if there was a difference in response based on audience segment as well.
What messaging will get a more positive response for Giving Tuesday Now in an email campaign?
|Treatment Name||Click Rate||Relative Difference||Confidence|
|C:||Giving Tuesday Now Email - Mission Message First||1.5%|
|T1:||Giving Tuesday Now Email - Campaign Message First||1.6%||8.9%||85.8%|
This experiment has a required sample size of 69,330 in order to be valid. Since the experiment had a total sample size of 77,990, and the level of confidence is not above 95% the experiment results are not valid.
While we weren’t able to validate the results based off of donations, we did see a directional lift in clicks from the treatment that used Giving Tuesday Now messaging at the top of the email for donors. What is really interesting is that nondonors performed just opposite – that segment had a 9% increase in clicks when the missional language was used first in the email. What we can learn from this experiment is that donors are already convinced and sold on the mission of the organization – after all, they are already donating to that organization. So when the donor file saw a special giving opportunity, this resonated with them and they responded. For nondonors, being presented with copy that talked about Giving Tuesday Now may have been a turn off to them. Being communicated about a giving opportunity right away, when they haven’t yet converted to becoming a donor for the organization caused friction and less people responded. This learning is critical for us as we move forward into other giving campaigns as we try and understand the different motivating factors for each audience segment – not all audiences are going to be motivated by the same things.