357.7% lift How Facebook’s optimization options can impact donor conversion

Date Added: January 7, 2019 Research Partner: Buckner International Element tested: Advertising

Buckner International had been running their "7 Scriptures" offer for over a year and a half. In that time, we had run many different experiments in an effort to reduce the costs of acquiring both emails and donors. While these optimization efforts had been successful at reducing the cost per email, we had not been able to significantly impact the donor acquisition efforts.

Facebook, our primary advertising medium, had recently released a new feature called "Test and Learn." This allows us to do a true A/B split to measure the impact of the various advertising methods offered through their platform. With this new capability, we decided to test out a differ method for optimizing our ad spent. 

By default, Facebook will advertising to audiences based upon the primary conversion metric that is defined during setup. Historically, we had been optimizing our ads to find people most likely to download the offer. We decided to test optimizing for "purchases" which is a native event within the Facebook platform. A "purchase" in Buckner's case was a donation. We had learned that Facebook incorporates outside purchasing information when using their native event so, despite having much smaller conversion number, we wanted to see if Facebook would be able to successfully model a likely donor for the organization using this outside data.



View the experiment »



-20.5% drop How where the unsubscribe option in an email appeal impacts unsubscribe metrics

Date Added: January 4, 2019 Research Partner: EWTN Element tested: Email Design

With the volume of emails being sent during high-urgency campaigns, we normally see an up-tick in unsubscribes. In an effort to see if we could impact the unsubscribe rate, specifically on the last day of the year when we see the most unsubscribes, we hypothesized moving the unsubscribe button out of page view. We simply did this by just creating more space between the signature and the footer where the subscribe button lives. 



View the experiment »


-53.1% drop How the kind of image used on a homepage banner impacts donor conversion

Date Added: January 3, 2019 Research Partner: EWTN Element tested: Advertising

On EWTN's homepage slider, they used an image of Fr. Joseph (EWTN Chaplain & Chapel Dean, Postulant Director) and Michael Warsaw (EWTN Chairman and Chief Executive Officer) with a year-end message about continuing Mother Angelica's legacy through a donation to EWTN. We hypothesized that using an image of Mother Angelica might reinforce this message and also be a more familiar face that people are connected to and more motivated by to potentially make a gift. We tested these two images against each other during the final weeks of the year-end campaign.



View the experiment »


31.2% lift How adding an image in the email copy impacted clicks

Date Added: January 2, 2019 Research Partner: Leadership Institute Element tested: Email Design

The Leadership Institute wanted to see if including an image screenshot would impact clicks against an email with similar copy but no image. The image was a screenshot of an appearance of an LI employee on Fox News.

View the experiment »


Not Valid How utilizing statistics in an email appeal influenced results

Date Added: January 2, 2019 Research Partner: Buckner International Element tested: Email Copy

Buckner International was sending out their 2018 calendar year end campaign focused on the impact and need behind foster care and adoption. When discussing as a team, there was debate surrounding the use of statistics in the email appeals. There was published research indicating that statistics in an appeal could potentially lower response rates. We decided to test this theory with an email during the year end campaign.

View the experiment »



-19.4% drop How count-based urgency and percentage-based urgency affect conversion

Date Added: January 2, 2019 Research Partner: CaringBridge Element tested: Advertising

CaringBridge had been running sticky bar challenges throughout their year-end campaign, and had found a "power hour" method that created recurring urgency by setting hourly goals and inspiring donors to help reach them. As they neared the end of the campaign, they had a $25,000 matching gift they wanted to leverage to generate revenue, but decided to test the "donor-based" goal (with a number of donations displayed) against the traditional "goal-based" thermometer, which showed a percentage of the $25k gift remaining to be matched. Their hypothesis was that while thermometers have been tested to convey urgency, they might make the goal too abstract or "out of reach" for one single donor. They didn't want to go about "business as usual" without testing it.

View the experiment »


Not Valid How social pressure in the copy affects change in conversions

Date Added: December 31, 2018 Research Partner: Hillsdale College Element tested: Email Copy

This experiment was tested on Hillsdale College's online course page. They wanted to know if copy that pressured the reader to engage would perform better than the more encouraging treatment, which asked the reader to enhance what they already know. The control makes the reader feel like they need to be better or face ridicule, while the treatment encourages the reader. The hypothesis was that the more direct language would pressure the reader to make the conversion.

View the experiment »


56.9% lift How an expiring goal and increasing urgency affect donor conversion

Date Added: December 31, 2018 Research Partner: CaringBridge Element tested: Advertising

CaringBridge tried a new fundraising technique during the year-end campaign called the "Power Hour", where they would raise money in small hourly campaigns that expired in 60 minutes. As they were running this campaign (via a sticky bar on their site), they wanted to monitor what happened to clickthrough rate and conversion rate as the number of gifts dropped. They analyzed the data with four gifts remaining and the data with two gifts remaining to try to understand how an expiring goal affects donor conversion.

View the experiment »