How a question versus a problem at the beginning of an email affects click rates
KCBI
Experiment Summary
Ended On: 05/18/2021
KCBI was launching a new offering of a Verse of the Day email. This is an important offering and very inline with the organizations mission. In order to discern what motivates their readers to read on and to engage with the email we tested a beginning to the email that was a questions “What do you seriously need each morning?” against starting the email with the problem of mornings and finding time to spend with God, then offering the solution to making their mornings better. This was a 50/50 split test with the same subject line “What you need every day.”
Research Question
Introducing a problem at the beginning of the email will increase the readers interest in the solution
Design
Results
This experiment has a required sample size of 4,362 in order to be valid. Since the experiment had a total sample size of 6,912, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
The question at the beginning of an email increased the click rate by almost 12%. Our hypothesis is that the question started people on the path of thinking what is it that they need in the morning and then the solution was offered. Beginning with the problem, told them that they struggled to get what they need, rather than asking what they needed and engaging them in the options. The question also matched the subject line of “What you need every day” and so there was a smooth transition from the envelop of the email to the beginning of the conversation. The reader was more prepared for what came next. We also think there’s something to an upbeat tone when talking to KCBI’s audience.