How we evaluated the potential for email versus donor acquisition Experiment ID: #6406

Illinois Policy Institute

Experiment Summary

Timeframe: 3/17/2017 - 3/31/2017

In a previous experiment for Illinois Policy Institute, we had discovered the optimal tactic for email acquisition on their news website. By using a slide out newsletter offer, we were able to increase email acquisition by 470%.

However, email acquisition was one of several key performance indicators that the organization was looking to improve. Illinois Policy was also focused on finding ways to acquire donors from this same audience. With these two differing priorities, we decided to a launch a treatment that would focus on donor acquisition. The goal was to see how many email addresses would be given up by focusing on the donor acquisition element.

With this in mind, we launched an inline donation ask in the middle of various news articles. We called this the “Dear Reader” treatment and it talked about the nonprofit status of the organization and the other policy work that is being done in the state.

Research Question

Which offer (email vs. donor acquisition) will be more successful for the organization?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS


C: Email Slide Out
T1: Dear Reader


Treatment Name Conv. Rate Relative Difference Confidence
C: Email Slide Out 0.00%
T1: Dear Reader 0.03% 695.4% 98.0%

This experiment has a required sample size of 21,812 in order to be valid. Since the experiment had a total sample size of 61,378, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 695.4% increase in conversion rate
× 0% increase in average gift

Key Learnings

We discovered that the “Dear Reader” inline donation ask was able to boost the website’s donor conversion rate by 695%. There was always the ability to give a gift on the site but without the specific call out, visitors were not likely to take that action.

However, in prioritizing the donor acquisition over email acquisition, we discovered that IPI was giving up roughly 18 new email addresses for every donor that was acquired. When we compared this to the percentage of these new emails that eventually convert to a donor, we were then able to determine which offer would provide the most long-term value for the organization.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.

Pick your format of choice:

Related Experiments

This experiment is part of a series of experiments aimed at improving overall results. Take a look at some of the other iterations:

Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.