Not Valid The impact of using Facebook’s video creation tool on an ad campaign

Date Added: August 16, 2019 Research Partner: Care Net Element tested: Advertising

We've had a successful campaign advertising a pledge that led to a monthly donation page on Facebook. In a previous experiment, we found that using an image of a newborn baby on this offer significantly increased donor conversion. We decided to test that image against a new video using the video creation tool in Facebook. They hypothesis was that a 15 sec video would get more people to convert than the image.

View the experiment »


Not Valid How a radical redesign impacted the Shoes for Orphan Souls donation page

Date Added: August 16, 2019 Research Partner: Buckner International Element tested: Donation Page Form

Based upon the low traffic volume to the Shoes for Orphan Souls donation page, we knew that we would need to try something radical in order to validate an experiment. With this in mind, we decided to launch a radical redesign of the page in conjunction with a similar experiment on Buckner's main donation page. Both experiments tested similar elements:

  • We modified the form headers to be numbers steps instead of just labels.
  • We removed the gift array which we've seen help with organizations that have a large volume of mid-level and major donors
  • We increased the perceived security near the credit card form
  • We reiterated the gift size and impact near the donation button


View the experiment »


759.9% lift How reinforcing the value proposition for new subscribers through ad service during the welcome series increased downstream donor conversion rate

Date Added: August 16, 2019 Research Partner: Americans for Prosperity Element tested: Advertising

We wanted to experiment with the value of reinforcing Americans for Prosperity's value proposition for new subscribers as they participated in the New Subscriber Welcome Series to see if it would have any impact upon their program's ability to convert new subscribers into first time donors.

View the experiment »


26.8% lift How the kind and tone of cultivation emails sent can impact donor conversion

Date Added: August 16, 2019 Research Partner: Care Net Element tested: Email Copy

Care Net sends a weekly cultivation email to donors and nondonors on their file. They call this email their "weekly update" and it usually includes a couple of interesting articles or podcast episodes they encourage people to read or listen to. It's sent every Saturday morning from Ardee Coolidge - a familiar sender to their file as he is also normally the sender of appeal emails. We know that cultivation is critical to the success of an online fundraising program, but we questioned if we made it more personal in nature and changed elements of what was being sent, if we could increase engagement and ultimately donor conversion of their email file. 

To test this, we split their file in half - having an equal number of donors and nondonors in each segment, and ran a longitudinal test over a three month period that ended during their fiscal year-end. The control segment continued to receive the standard cultivation/communication and the treatment segment received similar content, but the approach was slightly different. We created a more intentional, high-touch email that used personal tone copy and presented people with one piece of content to read or listen to. We also changed the sender of the treatment emails over the three month duration of the experiment.



View the experiment »


-100.0% drop How using “lowest click” settings for donation remarketing decreased donor conversion rate

Date Added: August 16, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

In an attempt to experiment with what drives remarketing gifts for the recent Board Challenge campaign for The Heritage Foundation, we decided to experiment with attempting to drive more click volume by targeting remarketing ads based upon "lowest cost per click" value against the "purchase" optimization ad serving method. Our hypothesis is that if someone entered into the remarketing ads campaign, they clicked an email to go to the campaign landing page. So, instead of paying a premium to serve remarketing ads to those who are only most likely to "purchase" based upon Facebook's algorithm, but rather try to get as many people back to the donation page that saw it before, in an attempt to secure more donations overall.

View the experiment »


-41.1% drop How a radical redesign of the main donation form impacted giving

Date Added: August 16, 2019 Research Partner: Buckner International Element tested: Donation Page Form

Given the volume of traffic and the average gift size, we have struggled to validate an experiment on Buckner's main donation form in the past. With this in mind, we decided to test out a radical new donation form that changed multiple elements that we've seen work in the past with similar organizations. Specifically, we made the following changes:

  • We modified the form headers to be numbers steps instead of just labels.
  • We removed the gift array which we've seen help with organizations that have a large volume of mid-level and major donors
  • We increased the perceived security near the credit card form
  • We reiterated the gift size and impact near the donation button


View the experiment »


Not Valid How a radical donation page treatment impacts acquisition revenue

Date Added: August 16, 2019 Research Partner: Buckner International Element tested: Donation Page Form

After running for nearly six months, we had not been able to achieve a statistically significant lift to the donation page used with Buckner's 7 Scripture offer. After multiple iterative approaches, we decided to test a radical redesign of the page. We combined multiple factors based upon positive results seen with similar organizations. The down side to this approach is that it makes it difficult to isolate exactly what helped move the needle if a lift is achieved. However, the overall goal was to lift revenue and if that could be achieved then knowing the exact reason for the lift became less important.

View the experiment »


-32.0% drop How a softened message attached to the Freedom Pledge negatively impacted donor conversion rate

Date Added: August 16, 2019 Research Partner: Americans for Prosperity Element tested: Name Acquisition Copy

In 2019, Americans for Prosperity's development office received a new set of rules to prohibit the use of words in their communications that include "Freedom", "Liberty", "Big Government", and more than 30 others, the development team felt it was appropriate to properly test the negative impact of limiting their ability to use these cornerstone words.

View the experiment »


11.4% lift How decision friction impacts donor conversion

Date Added: August 16, 2019 Research Partner: Billy Graham Evangelistic Association Element tested: Donation Page Design, Donation Page Form

On BGEA's primary donation page, they offer people 14 different gift designations to choose from during the first step in their donation process. In an effort to optimize the giving experience and increase donor conversion, we previously tested removing all of the gift designation hypothesized narrowing down the number of gift designation options to the top five. Using historical giving data, we pulled the top five designations and developed a treatment that featured just those five. People would still be able to see the other 9 options if they wanted by clicking the "See more>>" link. We also developed another variation of the treatment that emphasized the top giving option as the "Most Urgent" need and removed the imagery and copy from the rest of the gift designations.

View the experiment »


Not Valid How the experience on a donation confirmation page impacts subsequent gifts and overall revenue

Date Added: August 16, 2019 Research Partner: Focus on the Family Element tested: Donation Page Headline, Donation Page Copy

A lot of focus gets put on the optimization and improvement of the actual donation process and little attention is put on the experience after a person makes a gift. We hypothesized that improving this experience and making the donor feel valued, celebrated, and affirmed in their decision to make a gift to Focus on the Family we might be able to impact and improve subsequent giving and overall revenue from the donor in the future. To do this, we developed a treatment experience on the donation confirmation page that thanked the donor by name, celebrated the donor in the copy we used, reinforced the value proposition of their gift by sharing two testimonials, and even had confetti explode on the page as soon as the donor landed on the page. We tested this approach for over six months to really assess the impact that we might be able to have on giving over a longer period of time. Half of the audience during those six months experienced the control confirmation page and the other half experienced the "Super, Happy, Thankful" treatment experience.

View the experiment »