Buckner International

How adding region-specific wording to an email affects click rate

Experiment ID: #8044

Buckner International

Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.

Experiment Summary

Timeframe: 11/28/2017 - 11/28/2017

Buckner International was giving a free offer to people in their email file as a gift for Giving Tuesday. They wanted to test whether adding region-specific copy in the email would increase clickthrough rate. The control had generic language, whereas the treatment would use contact-specific wording and apply their most local Buckner branch’s name in that copy.

Research Question

Does making copy more personal to the reader increase click rate in an email?

Design

C: Control (Not region-specific)
T1: Region-specific wording

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Control (Not region-specific) 0.29%
T1: Region-specific wording 0.30%2.7% 13.8%

This experiment has a required sample size of 3,550,558 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

The region-specific copy increased clicks by 3%, however, this experiment was not valid. A more radical test should be performed to increase learnings about conditional copy to the reader.


Experiment Documented by Allison Autrey

Question about experiment #8044

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.