How a change in sender influenced donations Experiment ID: #11472

Buckner International

Buckner International is a global ministry dedicated to the transformation and restoration of the lives we serve. We are a Christ-centered organization that delivers redemptive ministry to the most vulnerable from the beginning to the ending of life.

Experiment Summary

Timeframe: 1/22/2019 - 1/25/2019

Buckner International had historically seen success introducing a new sender in their solicitation emails. Their standard sender, Dr. Albert Reyes, is the President and CEO of the organization and his name is used for the majority of cultivation and solicitation emails. However, we had never directly tested the impact of a different sender as a head-to-head experiment. For the January solicitation, we decided to do that.

We split the file in half for both of the solicitation emails sent that month. We made sure to maintain the split so that we could measure the impact of the new sender on more than just the first email.

Research Question

Will utilizing an unknown sender significantly improve fundraising results for an entire campaign?

Design

C: Albert Reyes
T1: JoAnne Cole

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Albert Reyes 0.03%
T1: JoAnne Cole 0.06% 149.8% 99.3%

This experiment has a required sample size of 23,545 in order to be valid. Since the experiment had a total sample size of 87,910, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 149.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

The new sender led to increased email engagement and, subsequently, increase giving. With a 149.8% boost to donations, this is an important strategy to remember for future campaigns.

There is one thing to note: we saw an 85% increase in opens from the new sender for the first email in the series. This number fell to a 46% increase with the second email that was sent out. Both were significantly higher open rates than their standard sender but the decrease in open rate on the second send suggests it was just a temporary boost due to a new sender being used. We can use the “unknown sender” strategy in the future but it must be used sparingly or it will lose effectiveness.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.