How we are able to use subject lines to learn an audience’s affinities Experiment ID: #11330
Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.
Timeframe: 1/16/2019 - 1/20/2019
Buckner International was sending out their monthly cultivation email to their house file. In recent months, one of the primary discussion points related to their subscribers revolved around how we could increase their overall engagement. The open rate of the file had slipped in recent months.
In addition to evaluating technical solutions to the problem, we also wanted to see if we could identify topics that may reactivate people. Since many of these subscribers came from an offer related to scripture, we wanted to see if incorporating scripture earlier in the email (like the subject line) would increase their engagement.
Will using the scripture verse discussed in the email as the subject line increase email engagement?
|Treatment Name||Open Rate||Relative Difference||Confidence|
|C:||What is your destination in 2019?||9.2%|
This experiment has a required sample size of 728 in order to be valid. Since the experiment had a total sample size of 87,640, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
34.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
We saw a 34.8% increase in the open rate of the email when using the scripture verse as the subject line. We are evaluating the success of this experiment solely on open rate since there were no links to the website or possibilities to give. This is a great reminder of the power of utilizing scripture in our communications as a way to better relate to our own audience.
We will test this further in March by running a multi-email experiment using a segment that will receive solicitations with scriptural references and comparing their giving to a segment that does not have any scripture in their communications.