CaringBridge

How communal-based communication affects donor conversion from a matching gift popup

Experiment ID: #41417

CaringBridge

Experiment Summary

Ended On: 12/10/2020

CaringBridge was running a two-day match in the month of December as part of their year end campaign. They had previously tested into a highly effective popup during matches that used urgency and an hourly donor countdown to increase response during these time-sensitive campaigns. However, they wondered if more “communal” messaging that affirmed the donor in their relationship to the CaringBridge author and made a less direct ask would be more effective at getting people to enter the donation funnel.

They drafted a treatment that made some key shifts. First, it began by talking about the donor themselves—”You’ve been such a great friend by staying engaged through CaringBridge.” Then, it softened the ask by starting with “If it’s not too much to ask”, which gives the donor a sort of “release valve” from the pressure of “donate now”. Finally, the matching gift from the Board of Directors was positioned as an opportunity or service offering provided to the donor rather than something they were expected to respond to.

They split the traffic between the two popups and monitored downstream traffic separately to see how it affected both clickthrough rate and conversion, as they realized that the two do not always correlate.

Research Question

We believe that employing communal language for website visitors will achieve an increase in donor conversion rate.

Design

C: Control

T1: Treatment #1

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence
C:	Control	0.10%
T1:	Treatment #1	0.15%	58.3%	100.0%

This experiment has a required sample size of 30,697 in order to be valid. Since the experiment had a total sample size of 282,562, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

0% increase in traffic
× 58.3% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment with the communal language produced a 58.3% increase in donor conversion rate. This is significant in that donor conversion increased at a higher rate than clickthrough rate (see that data here), which meant that it not only produced more clicks, it produced more motivated clicks. There are a few reasons behind what might have contributed to the increase.

First, the intro of "You've been such a great friend by staying connected through CaringBridge" not only affirms the donor, it piques their interest. It's a compliment, rather than the presumptive "your donation will be matched" language on the control. This additional engagement could have contributed to more people reading the next sentence—"If it's not too much to ask, would you consider giving today so that our Board of Directors can match your gift?".

This next sentence presents the ask in a completely different way. First, the donor is given an initial choice: "is this too much to ask, or not?", before they read about the incentive. Second, the incentive is offered in a radically different way than the control. Where the control actually thanks the Board of Directors for matching, the treatment gives the donor the chance to be the hero by making the gift, and places the match in a position that is responsive to the donor's action.

This test illuminated the possibilities of "communal" language for CaringBridge—which is a very "communal" organization in the first place. This opens up the opportunity do to more testing on the donation page downstream from this ask.

Experiment Documented by Jeff Giddens

Jeff Giddens is President of NextAfter.

Question about experiment #41417

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.