International Mission Board

How removing organizational language in an email appeal impacts revenue

Experiment ID: #118841

International Mission Board

Experiment Summary

Ended On: 12/09/2022

For Giving Tuesday, the IMB sent out email appeals to their housefile. In one of the emails we wanted to test how removing organizational language would impact either donor conversion rate or overall revenue. We hypothesized by replacing “the IMB” with words like “us” or “our,” the prospective donor would feel like they’re giving to people instead of a organization or corporate entity.

Research Question

We believe that removing organizational language in an email appeal for prospective donors will achieve more revenue .

Design

C: Control
T1: Treatment #1

Results

 Treatment NameRevenue per VisitorRelative DifferenceConfidenceAverage Gift
C: Control $0.23$145.12
T1: Treatment #1 $0.3451.9% 87.8%$189.59

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.

Key Learnings

When pulling the results, the donor conversion rate was relatively the same. We saw a 16% increase in donor conversion rate but with only a level of confidence of 52%. We also saw little no impact in click-thru rate for these emails. However when looking at revenue data, we saw a 51% directional lift in revenue with a level of confidence of 87.8%. While this is not a statistically valid result, it can be surmised that there are some donors who may have been motivated to give a larger gift when reading the treatment copy. The treatment copy speaks to the donor as if they’re joining “us” to accomplish something as opposed to giving to someone–making this much more communal in nature.

It’s recommended we retest this concept and also include the donation page copy in the experiment so that the donation page copy also has the organizational language removed in the treatment experience.


Experiment Documented by NextAfter

Question about experiment #118841

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.