How identity language in a fiscal year end campaign email affects donor conversion
The Missionary Oblates of Mary Immaculate
The Missionary Oblates of Mary Immaculate are a Roman Catholic congregation of priests and brothers founded after the French Revolution by St. Eugene De Mazenod to work among the poor. Today there are nearly 4,000 missionaries working in more than 60 countries around the world.
Experiment Summary
Timeframe: 06/29/2021 - 06/29/2021
During Oblates fiscal year end campaign, we wanted to test identity language that clarified the identities of the people that would be helped by a donation to the campaign. Instead of saying, you will help people in Zambia, we told the story of one person in Zambia that would be helped. This sent on the second to last day of the campaign and was done as an A/B test in Mailchimp.
Research Question
Bringing clarity to who the donation helps will increase donor conversion.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Direct Appeal | 0.61% | ||
| T1: | Story-Based Appeal | 0.92% | 51.5% | 99.9% |
This experiment has a required sample size of 5,771 in order to be valid. Since the experiment had a total sample size of 37,899, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 51.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment increased donor conversion by 51%, which meant 23 more donations and $2,500 more dollars. We have a few hypotheses as to why this happened. First, it brought clarity to who the donation would help. It told the donor more of the story that they would be a part of. This also led to longer copy, that gave the donor more information, which could also have led to higher donor conversion.