How reinforcing the recurring gift option may have impacted single gift donor conversion – NextAfter
Emails

How reinforcing the recurring gift option may have impacted single gift donor conversion

Experiment ID: #21354

Emails

How do you get someone to open, click, and convert?

Experiment Summary

Ended On: 4/29/2020

As a part of the COVID-19 (Coronavirus) response efforts, American’s For Prosperity made available to concerned Americans an opportunity to add their name to an open letter to Congress as they planned a government response to provide citizens aid, as well as focused stimulus dollars to “restart the economy” again after the pandemic.

The open letter had become a wonderful engagement piece for the housefile, so they decided to open it up to prospective supporters to give them the opportunity to add their name alongside tens of thousands of other signers, as well.

Because the open letter was so popular, they decided to put a donation appeal behind it and test acquiring new donors, and did so very effectively.

But as time went on, they thought: “Would people activate a recurring gift if asked?” — so we decided to test it.

Research Question

Would presenting a reinforcement as to why the donor should consider a recurring gift impact donor conversion or revenue by single or recurring gift types?

Design

C: Control
T1: Treatment #1

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 4.0%
T1: Treatment #1 3.3% -16.1% 97.6%

This experiment has a required sample size of 6,672 in order to be valid. Since the experiment had a total sample size of 17,626, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 16.1% decrease in conversion rate
× 0% increase in average gift

Key Learnings

With 97.5% level of confidence, we identified that the single gift donor conversion rate decreased by -16.1% with this treatment.

With fewer donors coming through the single gift option, it’s worth noting that the average gift of those going through the treatment (and being exposed to the organization’s core value proposition in the recurring gift reinforcement message), we did notice an average gift that was 14% higher than the gifts collected in the control experience.

But how did this do overall at converting recurring donors?

It did slightly better—achieving a +39% increase in donors choosing the recurring gift option, and subsequently +90% more monthly recurring dollars, however neither of those metrics were validated in this study.

So, what is causing this decrease in single gift donor conversion? We actually believe that it is because the 50/50 traffic split is sending less motivated traffic to the treatment experience, rather than the control. However, the offer is time-relevant, and time is running out. With the need to maximize results before the opportunity is gone, we decided to pull the treatment and eliminate the potential threat to results that it was indeed the cause of decreased results.

Further experimentation with this treatment is advised when it has an opportunity to run for a longer period of time.


The Missionary Oblates of Mary Immaculate

The Missionary Oblates of Mary Immaculate are a Roman Catholic congregation of priests and brothers founded after the French Revolution by St. Eugene De Mazenod to work among the poor. Today there are nearly 4,000 missionaries working in more than 60 countries around the world.

Click on each one to dive into the details and why it’s important for your online fundraising.

Experiment Summary

Ended On: 6/9/2020

Missionary Oblates send out a monthly evangelize newsletter. They use the simpler format on all other emails that come from their organization as a way of creating a one-to-one relationship with the subscribers. They want to see if this same concept would apply to the stylized newsletter sent out each month. To do this, instead of breaking up all the stories, they put the entire first story into the email followed by the other links. The goal was the measure the impact on engagement (including giving).

Research Question

Would creating a simpler format for our Evangelize emails increase donations?

Design

C: Control
T1: Treatment #1

Result

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.02%
T1: Treatment #1 0.05% 171.4% 98.1%

This experiment has a required sample size of 26,789 in order to be valid. Since the experiment had a total sample size of 77,274, and the level of confidence is above 95% the experiment results are valid.

Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #885

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.