How communicating the impact of a recurring gift can cause friction to returning website visitors Experiment ID: #18681

Billy Graham Evangelistic Association

Experiment Summary

Timeframe: 9/1/2019 - 11/1/2019

On the BGEA primary donation page, we hypothesized that adding a line of copy communicating the impact of a recurring gift at the point of a person’s gift decision may influence a person’s decision to change their one-time gift to a recurring gift. No other elements were changed to the giving process.

Research Question

Can we increase recurring giving by  communicating the impact of a recurring gift at the point a person decides on the kind of gift to give?

Design

C: Control
T1: Recurring Gift Value Prop

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 2.6%
T1: Recurring Gift Value Prop 1.4% -46.2% 100.0%

This experiment has a required sample size of 1,018 in order to be valid. Since the experiment had a total sample size of 6,946, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 46.2% decrease in conversion rate
× 0% increase in average gift

Key Learnings

What we found is that the additional language on the gift type had no impact overall. But when we started looking into the different kind of traffic, the story was much different. For new donors to the BGEA website, this additional value proposition language increased recurring giving by 48%! At the same time, the added copy actually caused a decrease in recurring giving for returning visitors by 46%. This is a fascinating discovery in understanding the different motivations of site visitors. Because returning visitors are used to the website and how it looks and functions, the additional recurring gift value proposition caused friction in the giving process resulting in fewer donations.


Experiment Documented by...

Courtney Gaines

Courtney is the Senior Director of Optimization at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.