How the sender of cultivation emails during a campaign performed when soliciting for gifts
Leadership Institute
Experiment Summary
Ended On: 01/06/2021
The Leadership Institute decided to experiment with adding donor stories on top of their regularly scheduled calendar year-end campaign email appeal schedule to experiment with whether or not it inspired more people to give at all or increased generosity from those that saw the donor story emails.
As the second part of this year-end experiment, we were interested to learn whether the second sender (who sent the donor stories) would increase the donor conversion rate and/or revenue when soliciting gifts from the treatment segment (that saw the donor stories) over that of the control segment (that did not).
Research Question
We believe that the sender of cultivation emails during a campaign for campaign recipients will achieve increase revenue.
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | $1.05 | $109.61 | ||
| T1: | Treatment #1 | $0.92 | -12.3% | 61.1% | $111.28 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.
Key Learnings
With a 61.1% level of confidence, we observed a -12.3% decrease in revenue.
Similarly, we only observed with a 48.7% level of confidence a -13.6% decrease in donor conversion rate, as well.
Ultimately, we were unable to validate either metric through this experiment and further experimentation may be required.