How we are able to use subject lines to learn an audience’s affinities Experiment ID: #11330

Buckner International

Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.

Experiment Summary

Timeframe: 1/16/2019 - 1/20/2019

Buckner International was sending out their monthly cultivation email to their house file. In recent months, one of the primary discussion points related to their subscribers revolved around how we could increase their overall engagement. The open rate of the file had slipped in recent months.

In addition to evaluating technical solutions to the problem, we also wanted to see if we could identify topics that may reactivate people. Since many of these subscribers came from an offer related to scripture, we wanted to see if incorporating scripture earlier in the email (like the subject line) would increase their engagement.

Research Question

Will using the scripture verse discussed in the email as the subject line increase email engagement?

Design

C: What is your destination in 2019?
T1: Acts 27

Results

Treatment Name Open Rate Relative Difference Confidence
C: What is your destination in 2019? 9.2%
T1: Acts 27 12.3% 34.8% 100.0%

This experiment has a required sample size of 728 in order to be valid. Since the experiment had a total sample size of 87,640, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    34.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

We saw a 34.8% increase in the open rate of the email when using the scripture verse as the subject line. We are evaluating the success of this experiment solely on open rate since there were no links to the website or possibilities to give. This is a great reminder of the power of utilizing scripture in our communications as a way to better relate to our own audience.

We will test this further in March by running a multi-email experiment using a segment that will receive solicitations with scriptural references and comparing their giving to a segment that does not have any scripture in their communications.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.