31.1% lift How increasing clarity in the value proposition and creating differentiation in the call-to-action affects new site signups

Date Added: July 3, 2019 Research Partner: CaringBridge Element tested: Name Acquisition Copy

CaringBridge wanted to increase the number of new journal sites that were created from the homepage, primarily from mobile devices (which constituted the majority of traffic). After an initial review of the homepage, a few potential issues were identified. First, new site signups were not prioritized—"visiting a friend" was above it in the visual hierarchy. On mobile, the headline "Start your free website" and the call to action "Start a site" were right next to each other, with no description in between. In fact, most of the value proposition was actually below the call to action. They decided to create two treatments— The first treatment used a visual image of someone using a computer to access CaringBridge—to reinforce the feeling that a CaringBridge site creates. Second, a new call-to-action was used: "Create a site", instead of "Start a site", to emphasize that the site was new. Finally, the headline was changed to clarify what CaringBridge offered—"Personal, protected websites for every health journey". This copy was on the treatment, but was secondary to the headline "Share, Connect and Rally Support". They wanted to also see if "help text" would further increase conversions. They knew this would vary based on the visitor's familiarity with CaringBridge, but they added additional bullet points of copy, as well as personalizing the headline to reflect what CaringBridge can do for you, the visitor: "Start your free, personal CaringBridge site today". They launched a three-way test to determine which experience drove the most new site signups from the homepage.

View the experiment »


35.7% lift How a video ad increased email acquisition rate

Date Added: July 3, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

As a part of The Heritage Foundation's acquisition efforts promoting their "What You Need to Know about Religious Freedom" ebook, we wondered what impact a video-based ad would have on our ability to acquire emails and donors.

View the experiment »


Not Valid How the placement of a donation ask in an article impacts donor conversion

Date Added: June 27, 2019 Research Partner: EWTN Element tested: Advertising

On the National Catholic Register's website, we had previously tested adding a donation opportunity within the site's articles. We found we were able to increase donor acquisition by 800% with this "Dear Reader" style donation ask. We then rolled this out to all traffic and monitored engagement metrics on the website. Engagement for the articles didn't drop off and we continued to see donations come in from the feature. We then hypothesized the placement of the Dear Reader. Would we be able to increase donor conversion by placing it mid-article or at the end of the article? We tested this by splitting the traffic 50/50.

View the experiment »


23.8% lift How visually emphasizing a gift amount on a primary donation page impacts revenue

Date Added: June 27, 2019 Research Partner: The Missionary Oblates of Mary Immaculate Element tested: Donation Page Form

Significant testing had been done on the Missionary Oblates primary donation page in an effort to increase donor conversion. Because of the low conversion rate from this page, we weren't able to validate these experiments or help improve conversion on the page. Because of this, we hypothesized testing an element on the page that might impact overall revenue and average gift. For this experiment we tested visually emphasizing a larger gift amount than what we knew was the average gift on this page. To do this, we put the higher suggested gift amount in the middle of the gift arrays, put a box around it and noted that gift amount to be the "most popular".

View the experiment »


Not Valid How an additional design element affected registrations from a webinar email

Date Added: June 27, 2019 Research Partner: NextAfter Element tested: Name Acquisition Copy

The majority of our emails are hyper-personal and text only. But recently, we've seen digest-style emails perform better using some basic design elements including simple images and HTML buttons. We believe the reason for this is that our emails can be very heavy. And if you're not reading every line, a design element can quickly show what the offer is and provide a clear call-to-action. In this case, we wondered if a similar approach could help improve registration rates in our webinar invitation emails.

View the experiment »


-39.5% drop How creating a longer “process” impacted desktop donor conversion

Date Added: June 27, 2019 Research Partner: Alliance Defending Freedom Element tested: Donation Page Copy, Name Acquisition Form

In an effort to improve downstream donor conversion of people signing the statement of belief on the ADF blog, we decided to test out a new way to frame the process. The current control had the first step of signing the statement of belief and then the second step was a thank you page with a donation form on it. Our hypothesis was that we could improve donor conversion by making it feel more like a single process with multiple steps than two unique processes. To do this, we added a "Step 1 of 2" on the statement of belief and then modified the donation page to make it feel like the second step rather than a complete new page.

View the experiment »


30.9% lift The impact of a membership offer on organic website visitors

Date Added: June 27, 2019 Research Partner: Alliance Defending Freedom Element tested: Advertising, Donation Page Copy

In the continued efforts to measure the potential impact of membership on the ADF audience, we devised an experiment to be run on the homepage of the main website. For the treatment version, we added a small ad below the main website banner and an additional navigational button to "Become a Member." For the control version, we left the navigation the same and added an ad below the main website banner for an acquisition page that was comparable* to the membership page. 

To validate this experiment, we targeted all visitors landing on the home page that did not have a recent gift. To be consistent, once an individual was in the treatment group, they would continue to see the "Become a Member" navigational option as they moved through the site.

* The control acquisition page was comparable in that it consisted of several stories about ADF and its clients, had similar imagery, and culminated in an ask for a donation. The page used was the "Stories of Faithfulness" offer.



View the experiment »


119.3% lift How framing the opposition in familiar terms affects donor conversion

Date Added: June 7, 2019 Research Partner: Leadership Institute Element tested: Email Copy

The Leadership Institute has long raised money to train conservatives to beat “leftists.” This term was chosen over the more common “liberal” since “leftist” was believed to be more specific, more powerful, and convey more emotion. Moreover, “liberal” could be imprecise, referring in some contexts to “classical liberals” who often align with conservatives. They wondered, though, whether these assumptions were correct. Is “liberal” in fact a better understood term than “leftist,” and might it convey more emotion? They decided to run a test to see if the more commonly used "liberal" would increase response. Given relatively small sample sizes, they knew it would take several email appeals to determine a winner, and they also knew they didn't want to validate it on a single email. They set up three email appeals, with "liberal" and "leftist" being the only words they interchanged.

View the experiment »


Not Valid How including a small, tangible goal impacted overall donor conversion

Date Added: June 7, 2019 Research Partner: Buckner International Element tested: Email Copy

Buckner International was running an appeal around providing scholarships to foster children in order to go to camp. In the past, we had observed increased results when we used small, tangible goals in our appeal but it had never been tested directly. We decided to run a head-to-head test for this campaign to understand the impact of using these small goals with the Buckner donor file. For this campaign, we split the full house file in half and send three total emails to each segment. The control segment would not have a goal and the treatment segment would have a goal of sending 10 children to camp. We would then combine the giving across all three messages to validate the results.

View the experiment »


305.4% lift How a membership ask to new visitors to The Daily Signal increased donor conversion rate

Date Added: June 7, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

Recently, we found that displaying the "core ask" for supporting The Daily Signal to "returning" visitors to the donation microsite increased results over a traditional membership ask. In phase 2 of this experiment, we decided to further experiment with a "membership ask" vs. a "core ask" to support The Daily Signal to "new" and "returning" visitors.

View the experiment »