18.3% lift How relevant landing page copy impacts email acquisition rate

Date Added: August 21, 2017
Research Partner: Hillsdale College
Element tested: Name Acquisition Copy

Hillsdale College knew that their course “Western Heritage,” had a great value proposition. They struggled, however, to find a way to adequately describe this course and its importance on their course signup page. They wondered if tying this course material to a current event would increase their course signups. To test, they created a treatment page with landing page copy that described a recent speech given by President Trump.



View the experiment »


-7.3% drop How the presence of course completion time impacts email acquisition

Date Added: August 18, 2017
Research Partner: Hillsdale College
Element tested: Name Acquisition Copy

Hillsdale College often gives a short description on their course signup page of what to expect when taking a specific course. They wondered if the time commitment of a course impacted the motivation of the user. By launching an A/B test, they introduced a treatment page that clearly stated how many hours the user would need to commit to finishing a particular online course–in this case, “An Introduction to C.S. Lewis: Significance and Writings.”



View the experiment »


37.2% lift How an intentional focal point affects email acquisition rate on a Facebook Ad

Date Added: August 18, 2017
Research Partner: Hillsdale College
Element tested: Advertising

Hillsdale College reviewed the Facebook ad creative for their online course “Winston Churchill and Statesmanship,” and found that their ad clickthrough rate was below the normal threshold. They wondered: Will a more focused Facebook ad image increase both clickthrough and email acquisition rate? They created a second version of their Winston Churchill Facebook ad that (1) added a lighter color to the background, (2) blurred the background and (3) increased the size of the supporting ad text. They set this treatment against the control and began to monitor the results.



View the experiment »


22.0% lift How additional “personal” value proposition in advertising copy affects email acquisition rate

Date Added: August 18, 2017
Research Partner: The Heritage Foundation
Element tested: Advertising

The Heritage Foundation ran a campaign demanding that the Senate “do their job” and install judges in federal court vacancies. They wanted to provoke a powerful emotional response in their advertising, and developed two ad treatments to test against each other. The first stated the premise of the campaign clearly: there are 120 federal court vacancies that must be filled. It then continued to make an emotional appeal that the American people and the President had done their job, but the Senate refused to do theirs.

The treatment started with a first-person-plural statement: “We are more than a little frustrated right now.” It then went on to detail how things should work, as opposed to how they are currently (not) working. This brought a personal edge to the copy, which was intended to make it feel less like an ad and more like an actual Facebook post. They hypothesized that this would increase engagement, and when combined with the emotional value proposition, would produce a higher petition signature conversion rate.



View the experiment »


Not Valid How direct copy affects donation page conversion rate

Date Added: August 18, 2017
Research Partner: CaringBridge
Element tested: Donation Page Headline, Donation Page Copy

CaringBridge’s primary donation page had proven to show better results with a broader value proposition, as opposed to the highly personalized value proposition presented on their tribute donation pages.

In a copy review of their tribute donation page, they wondered if the question posed in the headline: “What does CaringBridge mean to you?” added cognitive friction as people tried to answer this question for themselves, instead of continuing to ingest the value proposition.

They created a treatment that instantly offered an opportunity to the reader—”You make CaringBridge better.” The treatment copy removed questions and made statements, telling them that if they valued CaringBridge, their donation was vital.

They launched an A/B test to determine a winner.



View the experiment »


274.5% lift How email engagement influence social media response rates

Date Added: August 18, 2017
Research Partner: Hillsdale College
Element tested: Advertising

Hillsdale College was promoting the launch of their new DVD version of their popular Constitution 101 course. They had sent an email to their house file and wanted to see if they could raise more money by remarketing to that audience. Using a new technology named LeadsBridge, they created two distinct custom audiences in Facebook: one of subscribers that had clicked on the email and another of those that had simply opened.

Using these two distinct audiences, they then served them the same Facebook ad with a direct ask for a gift in order to receive the DVD set.

They closely monitored results to see which group was more likely to convert, and therefore should receive more remarketing dollars.



View the experiment »


29.8% lift How length of ad copy in a Facebook ad affects conversion rate

Date Added: August 18, 2017
Research Partner: FamilyLife
Element tested: Advertising

FamilyLife was promoting a name acquisition offer surrounding a free ebook on Facebook. They wanted to see if longer, value proposition-centric body copy or a shorter, to-the-point ad copy would increase conversion rate. All other elements of the ad remained the same.



View the experiment »


Not Valid How a more personal copy treatment affects conversion rate on an acquisition offer

Date Added: August 17, 2017
Research Partner: Buckner International
Element tested: Name Acquisition Copy

Buckner International was running a name acquisition campaign tied to a free ebook offer. They wanted to test if more personal, emotionally-charged body copy would affect conversion rate on the offer landing page. We also removed a scripture reference and a bolded portion of copy from the body. All other elements remained the same.



View the experiment »


271.6% lift How a program-specific value proposition impacted donor conversion

Date Added: August 17, 2017
Research Partner: The Heritage Foundation
Element tested: Donation Page Design, Donation Page Headline, Donation Page Copy

During The Heritage Foundation Board Match fundraising campaign, we wanted to incorporate one of their more popular programs (The Daily Signal) into a fundraising appeal email. The goal was to show the broad impact of the organization by talking about the various aspects of the organization.

Historically, we had utilized the same landing page throughout the campaign for each email. This is something that we had tested into, finding that switching landing pages didn’t often result in any significant lifts so we determined it was not worth the extra time and energy.

However, in this situation, we felt that the brand recognition and popularity of the program may be worth creating its own page. We decided to test that hypothesis by creating a new landing page that was branded as The Daily Signal and focused its value proposition on that one aspect of Heritage.



View the experiment »


Not Valid How reduced email-specific landing page copy impacts email acquisition rate

Date Added: August 16, 2017
Research Partner: Hillsdale College
Element tested: Name Acquisition Copy

Hillsdale College discovered that creating channel-specific landing pages allowed them to launch tests and document test results with even greater accuracy. By splitting up the incoming traffic based on channel, they hoped they could match the experience closer to the motivation of the user. They hypothesized that individuals who received and read through an email offer about their online course, “Theology 101” may have a high enough motivation that they didn’t need to read a long value proposition on the landing page they clicked to visit. They launched an A/B test to find out.



View the experiment »