14.0% lift How explaining the content inside an email newsletter affects email acquisition rate

Date Added: November 8, 2019 Research Partner: Leadership Institute Element tested: Advertising

Leadership Institute’s website CampusReform.org had a sticky bar that showed for mobile devices and gave them the opportunity to sign up for their email list. The team wanted to test different options to see if one copy variant could increase conversion rate. The control had the headline “Don’t miss a story”, and offered CampusWire, the weekly newsletter. They decided to test two different angles. First, they moved the CampusWire name to the headline to see if introducing that earlier could pique curiosity about the newsletter and inspire people to sign up. In doing this, they realized that repeating the name in the short paragraph below the headline would be redundant, so they inserted new copy about leftist bias, in order to fill the space and continue to motivate the reader. Then, they created an unbranded, yet more specific headline in the style of the control: “Don’t miss a campus report”. They altered the paragraph beneath the headline to further explain the concept of a “campus report”, showing that it comes straight from student reporters and includes stories of leftist bias. They launched an A/B/C test to find a winner.



View the experiment »


55.9% lift Helpful language vs admonishment on a devotional landing page

Date Added: November 8, 2019 Research Partner: Care Net Element tested: Advertising

After our test on the Defending Your Pro-Life Beliefs course page where we found that conversational language directed at helping visitors outperformed more militant "Learn to defend your pro-life beliefs" language, we wanted to try a similar approach to our successful 40-Day devotional Prayers for Life. Our control version focused on the need to be more like Jesus and prayerfully interceding for the unborn at risk of abortion. The control copy focused on how we "can't afford" to stop praying and used heavier language to make visitors feel obligated to pray, and hopefully, download the devotional. Our variation used language directed at providing a "daily reminder" to pray to a person's inbox. The new copy assumed that visitors were already praying for the unborn, rather than assuming that they weren't. We tested this new approach on both the landing page and in the ad.

View the experiment »


-29.0% drop How adding a signer to a recurring ask popup affects clickthrough rate

Date Added: November 8, 2019 Research Partner: CaringBridge Element tested: Advertising

CaringBridge was running a popup for certain visitors that had previously visited a CaringBridge site. These visitors were shown the popup when they went to leave the site and it prompted them to become a recurring donor. We decided to run an experiment to see if making the ask relate more to an individual instead of the generic "website" would help increase traffic to the donation page.

View the experiment »


-98.5% drop How promoting a video, then retargeting video viewers with an offer decreased results

Date Added: November 8, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

Facebook agency partners consistently tell agencies, who then tell the clients they serve, that the best way to promote an offer is to launch with a video, optimize your campaign for video views, then retarget video viewers with an offer. 

The Heritage Foundation decided that they would like to test this process once and for all and get to the bottom of whether or not this approach does indeed increase results.



View the experiment »



65.1% lift How the cover image of an ebook increased results

Date Added: November 1, 2019 Research Partner: The Fund for American Studies Element tested: Advertising

The Fund for American Studies launched a new acquisition offer and after running it for a month and a half, the team needed to improve ad clickthrough rate and email signup rates to boost performance for the offer.

View the experiment »


18.0% lift How personalization and gratitude affects conversion rate and average gift

Date Added: October 28, 2019 Research Partner: CaringBridge Element tested: Advertising

CaringBridge was running a matching grant as part of a fundraising campaign. They had previously tested and found that a sticky bar on the main donation page increases results, as it reminds donors of the limited-time match. They hypothesized that they could further increase results if they made that message come from a person (in this case, Kelly Espy, a CaringBridge employee), instead of appearing like a site-generated message. They also had Kelly’s message thank the donor for their support, presumptively showing gratitude for what they hoped the donor was about to do.

View the experiment »


Not Valid How a dropdown on a donate button affects donor conversion

Date Added: October 28, 2019 Research Partner: CaringBridge Element tested: Advertising

For Caringbridge, the primary navigation has always had a prominent donation button in the top right corner. This button took all potential donors to a generic donation page that offered both one-time and recurring giving options. There was a hypothesis that since a recurring donor and a one-time donor are two distinct segments, it may make sense to split this audience at the navigation level. Doing this would allow for targeted donation pages for each segment which may help increase the relevance and perceived value of the offer.

View the experiment »


-93.0% drop How a personal message of impact affects clickthrough rate on a recurring donation ask

Date Added: October 28, 2019 Research Partner: CaringBridge Element tested: Advertising

CaringBridge launched a new effort to acquire recurring donors. Since the most likely people to become recurring donors were people who had visited the site multiple times, they created an exit intent popup that fired as someone left CaringBridge after their second visit (and only their second visit). The first iteration of this popup was a "faceless" message that made a direct ask and sent the user to a recurring-only donation page. They hypothesized that making the ask from a mom who had used CaringBridge would increase clickthrough rate and get more people to the page, thus generating more recurring donors. They launched a test to determine a winner.

View the experiment »


51.3% lift How presenting an offer to people based on what they consume impacts conversion on parenting articles

Date Added: September 22, 2019 Research Partner: Focus on the Family Element tested: Advertising

Focus on the Family has done significant testing within the articles on their website. From testing the kind of offer to the placement of the offers, they have optimized and continue to optimize the conversion opportunities on the articles pages for the significant volume of traffic. Most recently they learned that presenting people a premium/book at the end of the articles, that readers could receive by giving a gift of any amount, had the most significant impact on donor conversion. To continue to test into this learning, they hypothesized that presenting people with a targeted offer based on their engagement on the Focus on the Family website might increase donor conversion. Focus identified the top premium offers based on what is most requested and the kind of content that people read and consume from their site and offered premiums that closely aligned with those topics. This approach was tested against offering people one single premium offer. We ran this experiment and found that a significant portion of the traffic on the article pages were either first time visitors to the Focus website, or this was the first article they consumed from Focus. This audience was not receiving targeted content offers based on cookies. Because of this, 68% of the donations from the treatment results had been receiving the single content offer/premium. 

After discovering this we wanted to re-test this approach with targeted content offers based on the current article they are reading. This small nuance to the experiment is critical in knowing whether a targeted content offer based on the kind of content people consume helps increases donor conversion. 



View the experiment »