25.1% lift How a donor-based appeal (instead of a goal-based appeal) affects conversion rate

Date Added: January 30, 2019 Research Partner: CaringBridge Element tested: Advertising

CaringBridge had a matching gift offer as part of their year-end campaign. They had recently developed a dynamic thermometer that would update as a goal was reached and had leveraged it to increase conversion. However, they wondered if the large goal amount was too vague, and possibly caused too much cognitive friction in the mind of the reader. Did it make the reader wonder if the large goal could be achieved? Did it make the reader do mental math to approximate how much had been raised? Most importantly—did it communicate to the reader that their gift could have an impact?

They decided to run an experiment with a donor-based appeal, breaking their day down into hourly goals and keeping a dynamic countdown of how many donors were needed to meet the hourly goal. Their hypothesis was that smaller goals would bring it more within reach of the donor, and that communicating a goal in terms of how many donations—rather than how much money—was needed would tell the donor how they could materially affect the goal. Notably, the treatment did not include the match language at all. 

They launched an A/B test to understand the impact of this treatment. 

View the experiment »

-19.4% drop How count-based urgency and percentage-based urgency affect conversion

Date Added: January 2, 2019 Research Partner: CaringBridge Element tested: Advertising

CaringBridge had been running sticky bar challenges throughout their year-end campaign, and had found a "power hour" method that created recurring urgency by setting hourly goals and inspiring donors to help reach them. As they neared the end of the campaign, they had a $25,000 matching gift they wanted to leverage to generate revenue, but decided to test the "donor-based" goal (with a number of donations displayed) against the traditional "goal-based" thermometer, which showed a percentage of the $25k gift remaining to be matched. Their hypothesis was that while thermometers have been tested to convey urgency, they might make the goal too abstract or "out of reach" for one single donor. They didn't want to go about "business as usual" without testing it.

View the experiment »

56.9% lift How an expiring goal and increasing urgency affect donor conversion

Date Added: December 31, 2018 Research Partner: CaringBridge Element tested: Advertising

CaringBridge tried a new fundraising technique during the year-end campaign called the "Power Hour", where they would raise money in small hourly campaigns that expired in 60 minutes. As they were running this campaign (via a sticky bar on their site), they wanted to monitor what happened to clickthrough rate and conversion rate as the number of gifts dropped. They analyzed the data with four gifts remaining and the data with two gifts remaining to try to understand how an expiring goal affects donor conversion.

View the experiment »

10.5% lift How encouragement language or an informative callout affect the number of comments left

Date Added: November 18, 2018 Research Partner: CaringBridge Element tested: Engagement

CaringBridge sought to increase engagement on their site by increasing the number of comments on journal posts. Not only did a comment increase the likelihood that someone would return, it actually provided a positive boost to the patient or author as well who was writing the journal. Their standard comment functionality just said "comments", and didn't have a true call to action. While someone could scroll to read the comments and there was a box, there was no incentive for them to actually leave a comment at the end of the post. The team developed two treatments: one that provided an "informative fact" that told them what the end result of the comment was to the reader: "Did you know? A quick comment, no matter the situation (positive or negative), shows your support". This was designed to encourage people to comment regardless of the news. Their second treatment was designed to get people into the comments, using the language "See comments", and then implicitly encouraged them to "Show [their] love and support" by doing the same. They launched the two treatments in an A/B/C test, monitoring

View the experiment »

39.3% lift Does a donation ask’s presentation (relative to other organic material surrounding it) impact generosity?

Date Added: November 2, 2018 Research Partner: CaringBridge Element tested: Advertising

Each CaringBridge user's private/protected site has a personal "homepage" where followers can quickly see all of the different types of information and actions related to that user's personal health journey. The tribute widget, an opportunity to give a gift to CaringBridge in honor of that user, was presented in the Control design (below) like an ad, where the rest of the content had a different format and organic feel. The preliminary research question was this: is the ad-like presentation / format of that tribute widget keeping people from actually taking it seriously?

View the experiment »

68.0% lift How ordering and value proposition affect donor conversion on a mobile device

Date Added: August 8, 2018 Research Partner: CaringBridge Element tested: Donation Page Form

CaringBridge had a different site that was used for mobile donation traffic, built before the site was made responsive. Over time, these donation pages did not absorb optimizations made to the desktop versions, where the majority of conversions occurred. They wanted to test the fully optimized version of the donation page in the mobile conversion experience, specifically from their JEN (Journal Email Notification) emails. The JEN mobile donation pages had an image at the top which presented potential friction, a short paragraph of non-personalized value proposition copy, and the donation form. The donation form did not have numbered steps, which had proven to increase conversion on the desktop pages. The team created two treatments—one with the form optimizations, and another with the form optimizations and the value proposition copy, so they could see the impact of each element.

View the experiment »

9.3% lift How visually realigning and streamlining a donation ask affects donations

Date Added: July 23, 2018 Research Partner: CaringBridge Element tested: Advertising

CaringBridge's tribute widget had long been the primary source of donation revenue. Since a site redesign, the widget had been located in the center of the site, anchored in a box that was offset from the rest of the page by an orange box. The team wondered if this visually removed the tribute widget from the flow of the site, and caused readers who might give to miss it entirely. They designed a treatment to left-align the widget, move it much closer to the copy, and only offset it from the copy by a simple orange line. The copy remained entirely the same.

View the experiment »

53.9% lift How additional tactics improved giving on top of an already powerful match appeal

Date Added: April 11, 2018 Research Partner: CaringBridge Element tested: Email Sender, Email Copy

In past experiments, we have discovered a powerful motivating factor: the family story. And it makes sense: to the donor, it makes the benefit more concrete and it is presented in a way that human beings in general are best likely to absorb and process (story). The lingering question, however, is to what end?  Is it possible to combine motivating factors like this family story with a match and to see an additive-like effect on total donations? Especially for email?

View the experiment »

-15.7% drop The effect of a reverse (high to low) gift array on donations

Date Added: April 3, 2018 Research Partner: CaringBridge Element tested: Donation Page Form

Gift arrays have always been a point of interest for fundraisers. We've tested different amounts, different options, different minimums. We've even eliminated the array in favor of an open field. What we have not tested is the order in which to present the amounts. Many nonprofits online, across all different verticals, when using a gift array choose to show the largest suggested amount first, rather than the lowest suggested amount, likely in hopes that donors will give more. Does this matter? Especially for CaringBridge donors, who have been quite responsive in helping us understand gift arrays to-date. How we test this: The control (original) keeps the standard rising suggested amount approach (start with the lowest ($50) on the left (assuming visitors in this case naturally read left to right) and then end with the highest ($250) The treatment, however, reverses this, by starting with the highest amount first in the eyepath (in this case, left).

View the experiment »

78.4% lift How increasing the clarity of the value proposition affects donor conversion rate

Date Added: March 12, 2018 Research Partner: CaringBridge Element tested: Advertising

JEN (or Journal Email Notification) emails are the lifeblood of the service that CaringBridge provides. These emails are sent out when an author creates a new post in a site, and CaringBridge sends out nearly one million of them each week. There is a donation ask in these emails, but it had very low conversion. CaringBridge wanted to increase revenue from these emails by optimizing the language around the donation ask. The default language made a statement: Tribute donations are vital to keeping CaringBridge and [FirstName]'s website running. Honor [FirstName] with a donation to CaringBridge. They hypothesized that this copy did not present a clear value proposition to the donor because "tribute donation" is an internal term, and the call-to-action does not specify that the donation that they would honor their friend with is a tribute donation. They created four treatment versions to run weekly over a monthlong period to see if they could increase conversion rate. The first treatment didn't change the copy or CTA, but added a single line question that addressed the context for the ask: "Do you appreciate staying connected to [FirstName] like this?" The second treatment kept that introductory question, but changed the value proposition of the question to remind the user that CaringBridge was a free service, and asked them to make a generous donation in honor of their friend. The third treatment quantified the ask with some language that had proven a lift in other areas of the site by telling the user what their gift does, in tangible terms. Finally, the fourth treatment added a single piece of value proposition to the end of the third treatment—asking the reader to keep their friend's site online for a year.  CaringBridge launched a weekly testing schedule to determine a winner.

View the experiment »