How personalizing the Working Preacher fall campaign email effected the donor conversion rate - NextAfter
Luther Seminary

How personalizing the Working Preacher fall campaign email effected the donor conversion rate

Experiment ID: #76996

Luther Seminary

Experiment Summary

Ended On: 11/17/2021

This experiment was run on the first email sent to the housefile for the Working Preacher fall campaign.

The goal of this experiment was to increase donor conversion rate by personalizing the verbiage and simplifying the design of the email.

Within the treatment, we changed multiple factors including:

  • Beginning the email with a personalized story of how Working Preacher has positively affected a preacher’s job and messages
  • Explaining the why behind how coming up with new and interesting content, week-after-week, can be very difficult and strenuous
  • Simplifying the design and removing the templated “look” of the email to seem like it’s coming from a real person
  • Changing the wording from (in the control) speaking from an organization to (in the treatment) speaking in a conversation
  • Putting more emphasis on the donor and how important their donation is to this campaign
  • Adding more hyperlinks throughout the whole email to keep giving a donation at the top of mind for the potential donor

Research Question

We believe that personalizing an email for potential donors will achieve a higher donor conversion rate.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 1.6%
T1: Treatment #1 1.5% -5.6% 21.6%

This experiment has a required sample size of 146,100 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

We found that personalizing an email that is asking for donations to the Working Preacher fall campaign had no effect on the donor conversion rate.

We believe that Working Preacher’s donors are already accustomed to how Working Preacher talks throughout their campaigns, so personalizing the email did not have an effect on the donor’s motivation to give.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #76996

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.