194.6% lift How purchase optimization affects donor conversion in a Facebook campaign

Date Added: March 2, 2019 Research Partner: Hoover Institution Element tested: Advertising

Hoover Institution was running a Facebook campaign to promote their Intellectual Poverty offer, and wanted to test the effectiveness of optimizing for Facebook's native Purchase event, which brings in outside financial data to inform targeting, on instant donor conversion. The control version of the campaign optimized for a custom conversion (ebook signup). Both campaigns were identical in ad set targeting and content.

View the experiment »


78.4% lift How increasing the clarity of the value proposition affects donor conversion rate

Date Added: March 2, 2019 Research Partner: CaringBridge Element tested: Email Copy

JEN (or Journal Email Notification) emails are the lifeblood of the service that CaringBridge provides. These emails are sent out when an author creates a new post in a site, and CaringBridge sends out nearly one million of them each week. There is a donation ask in these emails, but it had very low conversion. CaringBridge wanted to increase revenue from these emails by optimizing the language around the donation ask. The default language made a statement: Tribute donations are vital to keeping CaringBridge and [FirstName]’s website running. Honor [FirstName] with a donation to CaringBridge. They hypothesized that this copy did not present a clear value proposition to the donor because “tribute donation” is an internal term, and the call-to-action does not specify that the donation that they would honor their friend with is a tribute donation. They created four treatment versions to run weekly over a monthlong period to see if they could increase conversion rate. The first treatment didn’t change the copy or CTA, but added a single line question that addressed the context for the ask: “Do you appreciate staying connected to [FirstName] like this?” The second treatment kept that introductory question, but changed the value proposition of the question to remind the user that CaringBridge was a free service, and asked them to make a generous donation in honor of their friend. The third treatment quantified the ask with some language that had proven a lift in other areas of the site by telling the user what their gift does, in tangible terms. Finally, the fourth treatment added a single piece of value proposition to the end of the third treatment—asking the reader to keep their friend’s site online for a year.  CaringBridge launched a weekly testing schedule to determine a winner.

View the experiment »


Not Valid How Facebook ads impacted a campaign’s non-member conversion rates

Date Added: March 2, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

During The Heritage Foundation's Kickoff Campaign, we wanted to run Facebook ads designed to remind members to activate their membership to see if it would boost response rates by adding a 3rd channel (digital ads) along side regularly scheduled email and direct mail messages.



View the experiment »


48.4% lift How a long-form story in an email affects webinar registrations

Date Added: March 2, 2019 Research Partner: NextAfter Element tested: Email Copy

In this experiment, our original email was trying to communicate the importance of a/b testing by telling a personal story. This is typical of many emails that we write – we often look for a personal story to make help the recipient see that we're humans (not just marketers) and use that connection to lead into a content offer of some type. But we wondered if it might be more effective to tell this same story in a much shorter format, and spend more time talking about the specifics of what someone will learn in the webinar, rather than solely relying on the landing page to cover those details.

View the experiment »


Not Valid How Facebook ads impacted a campaign’s lapsed member conversion rates

Date Added: March 2, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

During The Heritage Foundation's Kickoff Campaign, we wanted to run Facebook ads designed to remind lapsed members to renew their annual membership to see if it would boost response rates by adding a 3rd channel (digital ads) along side regularly scheduled email and direct mail messages.



View the experiment »


Not Valid How using a button instead of a raw link impacts conversion in a cultivation email

Date Added: March 2, 2019 Research Partner: The Missionary Oblates of Mary Immaculate Element tested: Email Call-to-Action

The Missionary Oblates had tested into using raw links for the hyperlinks in their email communication. This finding was over a year ago and they have continued to use raw links in all of their emails since then. As they were sending a cultivation email to their email file they hypothesized that maybe their file had grown used to the raw link and it wouldn't have the same impact as it had when they had originally tested it. They developed a treatment that used a button instead of a raw link and split their file in half to see the impact on clicks.

View the experiment »


Not Valid How a “forwarded” email impacts conversion

Date Added: March 2, 2019 Research Partner: Focus on the Family Element tested: Email Design, Email Copy

Focus on the Family ran a year end campaign in 2018. They hypothesized that sending an email that looked like a "FWD" would increase engagement and conversion. They decided to test this against an email with a subject line that didn't have "FWD" in it.

View the experiment »


14.9% lift How a more aggressive messaging angle impacts email acquisition

Date Added: March 2, 2019 Research Partner: Alliance Defending Freedom Element tested: Name Acquisition Copy

Jack Phillips, a client of Alliance Defending Freedom, had two significant events happen to him in a short window of time. After his Supreme Court victory, he was attacked again by the State of Colorado for a situation remarkably similar to the original case. As a result of this attack, Jack decided to take the fight back to the state and is suing them for the continued harassment of his religious freedom. With these two events happening around the same time, we wanted to find out which angle the ADF subscribers would find most appealing. To do this, we created a two statements for visitors to add their name to. One took the standard approach of "Defending Religious Freedom" while the other was far more aggressive and focused on "Fighting Back."

View the experiment »


21.4% lift How a plain text email performed against an email with a logo banner

Date Added: March 2, 2019 Research Partner: Focus on the Family Element tested: Email Design

Focus on the Family ran a year end campaign in 2018. They hypothesized that stripping down the "Last chance" emails to look more like Outlook or plain text emails would perform better than the control email which used large banner logos and video screenshots. They tested the outlook styles between two different signers.



View the experiment »


-63.1% drop How increasing the gift array on a donor acquisition page impacted overall revenue

Date Added: March 2, 2019 Research Partner: The Heritage Foundation Element tested:

Continuous testing to improve revenue from clickthrough traffic from The Daily Signal’s “Dear Reader” visitors has yielded consistent wins by the control. At this point, we wanted to experiment with the gift array to see if we could increase the revenue by just simply asking for a higher gift amount.



View the experiment »