How the copy tone of an ad impacts conversion.
The Missionary Oblates of Mary Immaculate
The Missionary Oblates of Mary Immaculate are a Roman Catholic congregation of priests and brothers founded after the French Revolution by St. Eugene De Mazenod to work among the poor. Today there are nearly 4,000 missionaries working in more than 60 countries around the world.
Experiment Summary
Timeframe: 04/04/2017 - 04/18/2017
The Missionary Oblates offered a special Easter Novena to people through paid Facebook ads. They wanted to test the motivation of the end-user by testing the tone of the copy and how they presented the offer. The control ad presented the offer early on in the copy and had a more uplifting tone with an invitation to get the offer. The treatment talked about the importance of Easter and encouraged people to get the Novena.
Research Question
Which ad would acquire more email addresses?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Oblates copy | 1.0% | ||
| T1: | NA | 0.95% | -8.0% | 87.1% |
This experiment has a required sample size of 108,298 in order to be valid. Since the experiment had a total sample size of 133,187, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
This test didn’t quite validate and there was only a slight difference between the control and the treatment. The initial data showed the ad with a more uplifting tone that presented the offer earlier in the copy had a 8% increase in acquisition. We recommend that a more radical test be done using some of these same concepts from the winning variant in order to validate the experiment in the future.