14.9% lift How optimizing a page for the mobile experience impacts conversion

Date Added: September 22, 2019 Research Partner: EWTN Element tested: Name Acquisition Copy, Name Acquisition Form

The National Catholic Register offers a free trial of their newspaper circulation to people. The page that offers people the opportunity to sign up for a free trial gets a significant amount of mobile traffic, but the page is not mobile optimized. To see the impact of having a mobile friendly experience, we developed a treatment that optimized the page for mobile devices. The content and copy on the page remained identical to the treatment. 



View the experiment »


13.0% lift How minimal content impacts subscriber engagement

Date Added: September 22, 2019 Research Partner: Alliance Defending Freedom Element tested: Email Copy

The primary goal of Alliance Defending Freedom's weekly cultivation series is to engage subscribers with content that is central to ADF's mission. With this in mind, past optimization efforts have centered around finding the optimal tactics to lift open and click through rates. All of the emails are sent from the same sender, "Sarah", and are written to appear as one-to-one as possible. In recent discussions, a hypothesis was proposed that we could potentially increase the number of visitors clicking by reducing the amount of content we put into the email. The theory was based upon the thought that we may be spending too much time "selling" the content instead of letting the visitors get to the information. To test this hypothesis, we created a treatment version of the weekly email with a dramatically simplified message.

View the experiment »


803.8% lift How promoting an offer in the Dear Reader ad copy instead of membership increased donor conversion rate

Date Added: September 22, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

In previous experimentation with the "Dear Reader" ads for The Daily Signal, we had noticed that there was a consistent finding of 5x to 10x more click activity on the ad when we would change the call to action from "Activate your membership" to "Learn more." This told us that there was a lot of additional click activity available to us that we were not capturing. In an attempt to see if this would translate into increased giving and revenue, we experimented with promoting a new offer instead of membership in this ad for The Heritage Foundation.

View the experiment »


17.4% lift How using a suggested gift array on a primary donation page impacts donor conversion

Date Added: September 22, 2019 Research Partner: EWTN Element tested: Donation Page Form

On EWTN’s primary donation page, the donation form had historically used an open gift field. The open gift array is an option that has proven successful for certain types of organizations and in certain locations. However, we have also found that providing a gift array can help reduce decision friction which can increase conversion rate.

To determine what would work best for EWTN, we decided to set up an experiment on the primary donation form.



View the experiment »



Not Valid The impact of using Facebook’s video creation tool on an ad campaign

Date Added: August 16, 2019 Research Partner: Care Net Element tested: Advertising

We've had a successful campaign advertising a pledge that led to a monthly donation page on Facebook. In a previous experiment, we found that using an image of a newborn baby on this offer significantly increased donor conversion (https://app.winstonknows.com/experiment/2321)   We decided to test that image against a new video using the video creation tool in Facebook. They hypothesis was that a 15sec video would get more people to convert than the image. 



View the experiment »


Not Valid How a radical redesign impacted the Shoes for Orphan Souls donation page

Date Added: August 16, 2019 Research Partner: Buckner International Element tested: Donation Page Form

Based upon the low traffic volume to the Shoes for Orphan Souls donation page, we knew that we would need to try something radical in order to validate an experiment. With this in mind, we decided to launch a radical redesign of the page in conjunction with a similar experiment on Buckner's main donation page. Both experiments tested similar elements:

  • We modified the form headers to be numbers steps instead of just labels.
  • We removed the gift array which we've seen help with organizations that have a large volume of mid-level and major donors
  • We increased the perceived security near the credit card form
  • We reiterated the gift size and impact near the donation button


View the experiment »


759.9% lift How reinforcing the value proposition for new subscribers through ad service during the welcome series increased downstream donor conversion rate

Date Added: August 16, 2019 Research Partner: Americans for Prosperity Element tested: Advertising

We wanted to experiment with the value of reinforcing Americans for Prosperity's value proposition for new subscribers as they participated in the New Subscriber Welcome Series to see if it would have any impact upon their program's ability to convert new subscribers into first time donors.

View the experiment »


26.8% lift How the kind and tone of cultivation emails sent can impact donor conversion

Date Added: August 16, 2019 Research Partner: Care Net Element tested: Email Copy

Care Net sends a weekly cultivation email to donors and nondonors on their file. They call this email their "weekly update" and it usually includes a couple of interesting articles or podcast episodes they encourage people to read or listen to. It's sent every Saturday morning from Ardee Coolidge - a familiar sender to their file as he is also normally the sender of appeal emails. We know that cultivation is critical to the success of an online fundraising program, but we questioned if we made it more personal in nature and changed elements of what was being sent, if we could increase engagement and ultimately donor conversion of their email file. 

To test this, we split their file in half - having an equal number of donors and nondonors in each segment, and ran a longitudinal test over a three month period that ended during their fiscal year-end. The control segment continued to receive the standard cultivation/communication and the treatment segment received similar content, but the approach was slightly different. We created a more intentional, high-touch email that used personal tone copy and presented people with one piece of content to read or listen to. We also changed the sender of the treatment emails over the three month duration of the experiment.



View the experiment »


-100.0% drop How using “lowest click” settings for donation remarketing decreased donor conversion rate

Date Added: August 16, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

In an attempt to experiment with what drives remarketing gifts for the recent Board Challenge campaign for The Heritage Foundation, we decided to experiment with attempting to drive more click volume by targeting remarketing ads based upon "lowest cost per click" value against the "purchase" optimization ad serving method. Our hypothesis is that if someone entered into the remarketing ads campaign, they clicked an email to go to the campaign landing page. So, instead of paying a premium to serve remarketing ads to those who are only most likely to "purchase" based upon Facebook's algorithm, but rather try to get as many people back to the donation page that saw it before, in an attempt to secure more donations overall.

View the experiment »