How campaign first messaging impacts clicks from donors in Giving Tuesday Now emails Experiment ID: #21240

EWTN

Experiment Summary

Timeframe: 5/5/2020 - 5/5/2020

On Giving Tuesday Now, EWTN ran a campaign focused around acquiring recurring donors. Because Giving Tuesday Now is new to people, we were curious if that message would resonate with donors and nondonors on EWTN’s email file. We tested this by using mission based messaging at the beginning of the email and then later talked about Giving Tuesday Now for the control, and developed a treatment used Giving Tuesday Now focused messaging at the beginning of the email and then later used missional messaging. We tested this with two different segments – donors and nondonors to see if there was a difference in response based on audience segment as well.

Research Question

What messaging will get a more positive response for Giving Tuesday Now in an email campaign?

Design

C: Giving Tuesday Now Email - Mission Message First
T1: Giving Tuesday Now Email - Campaign Message First

Results

Treatment Name Click Rate Relative Difference Confidence
C: Giving Tuesday Now Email - Mission Message First 1.5%
T1: Giving Tuesday Now Email - Campaign Message First 1.6% 8.9% 85.8%

This experiment has a required sample size of 69,330 in order to be valid. Since the experiment had a total sample size of 77,990, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

While we weren’t able to validate the results based off of donations, we did see a directional lift in clicks from the treatment that used Giving Tuesday Now messaging at the top of the email for donors. What is really interesting is that nondonors performed just opposite – that segment had a 9% increase in clicks when the missional language was used first in the email. What we can learn from this experiment is that donors are already convinced and sold on the mission of the organization – after all, they are already donating to that organization. So when the donor file saw a special giving opportunity, this resonated with them and they responded. For nondonors, being presented with copy that talked about Giving Tuesday Now may have been a turn off to them. Being communicated about a giving opportunity right away, when they haven’t yet converted to becoming a donor for the organization caused friction and less people responded. This learning is critical for us as we move forward into other giving campaigns as we try and understand the different motivating factors for each audience segment – not all audiences are going to be motivated by the same things.


Experiment Documented by...

Courtney Gaines

Courtney is a Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.