How adding region-specific wording to an email affects click rate
Buckner International
Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.
Experiment Summary
Timeframe: 11/28/2017 - 11/28/2017
Buckner International was giving a free offer to people in their email file as a gift for Giving Tuesday. They wanted to test whether adding region-specific copy in the email would increase clickthrough rate. The control had generic language, whereas the treatment would use contact-specific wording and apply their most local Buckner branch’s name in that copy.
Research Question
Does making copy more personal to the reader increase click rate in an email?
Design
Results
Treatment Name | Click Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Control (Not region-specific) | 0.29% | ||
T1: | Region-specific wording | 0.30% | 2.7% | 13.8% |
This experiment has a required sample size of 3,550,558 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The region-specific copy increased clicks by 3%, however, this experiment was not valid. A more radical test should be performed to increase learnings about conditional copy to the reader.
Question about experiment #8044
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.