1,046.1% lift How ad targeting in Facebook affects instant donor conversion

Date Added: April 16, 2019 Research Partner: Buckner International Element tested: Advertising

Buckner was running a new offer called Discovering God's Will on Facebook using purchase optimization, and they wanted to test whether their traditional ad targeting methods were more effective than open targeting, which essentially tells Facebook to find purchasers from a much larger audience. The ads in both versions of the campaign were the exact same- the only change was in targeting method. 



View the experiment »


Not Valid The impact of Scripture in an appeal

Date Added: March 18, 2019 Research Partner: Buckner International Element tested: Email Copy

Buckner International had seen success with recent communications by leaning heavily into scripture. As a Christian organization, they would incorporate scripture verses into their cultivation emails and even had an acquisition offer that focused on scripture. With this in mind, we had the hypothesis that using scriptures in our appeals may increase the reader's sense of duty which would then increase the likelihood to respond. However, we wanted to be sure it wasn't isolated to a single email. To do this, we split the house file equally among donors and nondonors and maintained this split through the whole campaign. We then three distinct emails, each with a scripture and non-scripture, version to measure the impact on the gifts.

View the experiment »


-68.9% drop How the style of Facebook ad optimization impacts results

Date Added: February 25, 2019 Research Partner: Buckner International Element tested: Advertising

Buckner International had been seeing considerable success with their long-running devotional email acquisition offer. Experiments geared toward conversion optimization and the donation value proposition had led to significant boosts in donor acquisition. Since we were seeing wins by testing some of the more technical aspects of Facebook advertising, we decided to continue this series of experiments.

Our next hypothesis was related to the method of budget optimization. There are multiple ways to allocate media spend within Facebook's Ad Manager. We had previously tested setting a daily budget regardless of audience but that resulted in slight decrease in conversion rate. We wondered if removing the requirement for daily spending would help increase our results. To do this, we changed our budget optimization method to be based upon a Campaign Lifetime Budget.



View the experiment »


Not Valid How budget strategy in Facebook affects instant donor conversion

Date Added: February 18, 2019 Research Partner: Buckner International Element tested: Advertising

Recent experiments had resulted in a significant improvement in Buckner's active acquisition campaign known as "7 Scriptures." The greatest lift was the result of technical optimization of the Facebook ads. With this in mind, we wanted to take this one step further and continue our testing of the various advertising techniques available to us through Facebook. One of the ways ads are shown is based upon the method of bidding. Historically, we had set daily spending limits for each audience which equally spread the spend across each one. We had the hypothesis that if we allowed Facebook's algorithm to determine the allocation of spend, we may see better results. To test this, we set the budget at the campaign level, using Facebook's Budget Optimization tool instead of the ad set level (setting a daily limit per ad set). All ads and targeting remained the same. At the time of the test, each campaign was spending the same amount per day ($83 and change).

View the experiment »


149.8% lift How a change in sender influenced donations

Date Added: January 26, 2019 Research Partner: Buckner International Element tested: Email Sender

Buckner International had historically seen success introducing a new sender in their solicitation emails. Their standard sender, Dr. Albert Reyes, is the President and CEO of the organization and his name is used for the majority of cultivation and solicitation emails. However, we had never directly tested the impact of a different sender as a head-to-head experiment. For the January solicitation, we decided to do that.

We split the file in half for both of the solicitation emails sent that month. We made sure to maintain the split so that we could measure the impact of the new sender on more than just the first email.



View the experiment »


34.8% lift How we are able to use subject lines to learn an audience’s affinities

Date Added: January 20, 2019 Research Partner: Buckner International Element tested: Email Subject Line

Buckner International was sending out their monthly cultivation email to their house file. In recent months, one of the primary discussion points related to their subscribers revolved around how we could increase their overall engagement. The open rate of the file had slipped in recent months. In addition to evaluating technical solutions to the problem, we also wanted to see if we could identify topics that may reactivate people. Since many of these subscribers came from an offer related to scripture, we wanted to see if incorporating scripture earlier in the email (like the subject line) would increase their engagement.

View the experiment »


Not Valid How Facebook’s Bid Cap influenced overall results

Date Added: January 14, 2019 Research Partner: Buckner International Element tested: Advertising

Buckner International had recently experienced two significant lifts to their 7 Scriptures email acquisition offer that had resulted in a boost to instant donor conversion. The resulting boost also reduced the cost per donor. It fluctuated by day but were were seeing some days with a cost per donor acquired of $50 or less. However, it was unpredictable.

In an effort to control the costs, we decided to test out the setting a cap on our bid of $50. This would allow us to potentially limit our costs, assuming the strategy worked.



View the experiment »


Not Valid How the donation selection area can communicate tangible impact

Date Added: January 11, 2019 Research Partner: Buckner International Element tested: Donation Page Form

For Buckner International's Shoes for Orphan Souls fundraising campaign, we wanted to find the best way to visualize the kind of impact a gift will have. In this situation, $25 would fund a pair of shoes for a child in need. Historically, we had placed text on the donation buttons but the buttons were usually far lower than the average gift amount. We had the hypothesis that if we removed the buttons all together and created variable copy around the textfield that visitors entered their information, we would be able to secure larger gifts. We decided to test this out during the fundraising campaign.

View the experiment »


357.7% lift How Facebook’s optimization options can impact donor conversion

Date Added: January 7, 2019 Research Partner: Buckner International Element tested: Advertising

Buckner International had been running their "7 Scriptures" offer for over a year and a half. In that time, we had run many different experiments in an effort to reduce the costs of acquiring both emails and donors. While these optimization efforts had been successful at reducing the cost per email, we had not been able to significantly impact the donor acquisition efforts.

Facebook, our primary advertising medium, had recently released a new feature called "Test and Learn." This allows us to do a true A/B split to measure the impact of the various advertising methods offered through their platform. With this new capability, we decided to test out a differ method for optimizing our ad spent. 

By default, Facebook will advertising to audiences based upon the primary conversion metric that is defined during setup. Historically, we had been optimizing our ads to find people most likely to download the offer. We decided to test optimizing for "purchases" which is a native event within the Facebook platform. A "purchase" in Buckner's case was a donation. We had learned that Facebook incorporates outside purchasing information when using their native event so, despite having much smaller conversion number, we wanted to see if Facebook would be able to successfully model a likely donor for the organization using this outside data.



View the experiment »


Not Valid How utilizing statistics in an email appeal influenced results

Date Added: January 2, 2019 Research Partner: Buckner International Element tested: Email Copy

Buckner International was sending out their 2018 calendar year end campaign focused on the impact and need behind foster care and adoption. When discussing as a team, there was debate surrounding the use of statistics in the email appeals. There was published research indicating that statistics in an appeal could potentially lower response rates. We decided to test this theory with an email during the year end campaign.

View the experiment »