How removing organizational language in an email appeal impacts revenue
International Mission Board
Experiment Summary
Ended On: 12/09/2022
For Giving Tuesday, the IMB sent out email appeals to their housefile. In one of the emails we wanted to test how removing organizational language would impact either donor conversion rate or overall revenue. We hypothesized by replacing “the IMB” with words like “us” or “our,” the prospective donor would feel like they’re giving to people instead of a organization or corporate entity.
Research Question
We believe that removing organizational language in an email appeal for prospective donors will achieve more revenue .
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | ||
|---|---|---|---|---|---|---|
| C: | Control | $0.23 | $145.12 | |||
| T1: | Treatment #1 | $0.34 | 51.9% | 87.8% | $189.59 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.
Key Learnings
When pulling the results, the donor conversion rate was relatively the same. We saw a 16% increase in donor conversion rate but with only a level of confidence of 52%. We also saw little no impact in click-thru rate for these emails. However when looking at revenue data, we saw a 51% directional lift in revenue with a level of confidence of 87.8%. While this is not a statistically valid result, it can be surmised that there are some donors who may have been motivated to give a larger gift when reading the treatment copy. The treatment copy speaks to the donor as if they’re joining “us” to accomplish something as opposed to giving to someone–making this much more communal in nature.
It’s recommended we retest this concept and also include the donation page copy in the experiment so that the donation page copy also has the organizational language removed in the treatment experience.