Americans for Prosperity

How a recurring gift ask in a popup decreased donor conversion rates

Experiment ID: #11506

Americans for Prosperity

Experiment Summary

Timeframe: 04/19/2019 - 05/03/2019

Americans for Prosperity was looking for a way to increase recurring gifts to their “Torchbearer” program, so as a part of the strategy to position and promote the program, we helped them install a recurring gift popup that appeared when any donor on any fundraising form chose to give a gift (that was not already marked as a recurring gift). 

Research Question

Can a recurring gift popup, targeting single-gift donors, increase recurring donors and dollars, without negatively impacting single gift conversion rates and/or dollars?

Design

C: Control
T1: Recurring Gift Popup

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 5.1%$0.00
T1: Recurring Gift Popup 1.6%-68.2% 99.8%$0.00

This experiment has a required sample size of 215 in order to be valid. Since the experiment had a total sample size of 955, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 68.2% decrease in conversion rate
× 0% increase in average gift

Key Learnings

Although we saw a 68.2% decrease in donor conversions, with a 99.7% level of confidence, we learned a valuable lesson. 

That lesson is: Recurring gift popups drastically decreased donor conversion rates when running on acquisition landing page forms. The majority of the difference between the control and treatment segments in the data shown as a part of this experiment was actually produced from those acquisition offer pages.

When removed from the sample, we actually generated a lift in recurring donors through traditional donation form funnels regularly available on the website (which, that single gift alone will cover the revenue difference created between these two segments of this experiment).

It is advised that we do not use recurring gift popup treatments on acquisition pages moving forward. Our next step is to properly isolate the primary donation funnels of the website (less any acquisition forms), then record the experiment again.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #11506

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.