How a change in sender influenced donations
Buckner International
Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.
Experiment Summary
Timeframe: 01/22/2019 - 01/25/2019
Buckner International had historically seen success introducing a new sender in their solicitation emails. Their standard sender, Dr. Albert Reyes, is the President and CEO of the organization and his name is used for the majority of cultivation and solicitation emails. However, we had never directly tested the impact of a different sender as a head-to-head experiment. For the January solicitation, we decided to do that.
We split the file in half for both of the solicitation emails sent that month. We made sure to maintain the split so that we could measure the impact of the new sender on more than just the first email.
Research Question
Will utilizing an unknown sender significantly improve fundraising results for an entire campaign?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Albert Reyes | 0.03% | ||
| T1: | JoAnne Cole | 0.06% | 149.8% | 99.3% |
This experiment has a required sample size of 23,545 in order to be valid. Since the experiment had a total sample size of 87,910, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 149.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
The new sender led to increased email engagement and, subsequently, increase giving. With a 149.8% boost to donations, this is an important strategy to remember for future campaigns.
There is one thing to note: we saw an 85% increase in opens from the new sender for the first email in the series. This number fell to a 46% increase with the second email that was sent out. Both were significantly higher open rates than their standard sender but the decrease in open rate on the second send suggests it was just a temporary boost due to a new sender being used. We can use the “unknown sender” strategy in the future but it must be used sparingly or it will lose effectiveness.