8.8% lift How increasing the relevancy of a subject line impacts email open rate

Date Added: January 3, 2018
Research Partner: Hillsdale College
Element tested: Email Subject Line

Hillsdale College recently conducted a calendar year end campaign that focused on their unique independence from government funding. During the month of December, they sent appeals that focused on different initiatives they are advancing to secure education and restore constitutional liberty in America. In their fourth email appeal, they highlighted Imprimis–their popular monthly speech digest that has a wider readership than the Wall Street Journal. Before sending this appeal, they wondered: Will increasing the relevancy of the subject line increase email open rate? They created a treatment version with the words ‘Fake News’ and launched an A/B test to find out.



View the experiment »


Not Valid How a radical redesign impacts donor conversion during year-end.

Date Added: January 3, 2018
Research Partner: FamilyLife
Element tested: Donation Page Design, Donation Page Headline, Donation Page Copy

For FamilyLife’s year-end campaign, they had a special matching gift they offered to people in an effort to motivate more people to give during this critical fundraising campaign. The donation page was focused on this matching gift and encouraged people to give before December 31 to take advantage of this incentive. We hypothesized that while people will be motivated by this incentive, they might be more motivated if we increased urgency and reminded them of what their gift will be used for. We developed a radical redesign of the donation page that led with a FamilyLife value proposition, increased urgency with a countdown clock and also included the match incentive.



View the experiment »



Not Valid How adding value proposition to a headline on a homepage banner affects donation conversion rate

Date Added: January 3, 2018
Research Partner: FamilyLife
Element tested: Advertising

FamilyLife had a high-urgency banner focused on their 2 million dollar matching gift for their year-end campaign. They wanted to test whether a banner that led with value prop headline first instead of match language would increase donations more than a match-focused headline on banner.



View the experiment »


124.8% lift How value proposition impacts donor conversion for a serial ask

Date Added: January 3, 2018
Research Partner: Heritage Action for America
Element tested: Donation Page Copy

For Heritage Action, the serial ask shown to The Heritage Foundation online donors has become a significant source of revenue. It works by allowing Heritage donors to complete their gift and then immediately positioning a second ask for Heritage Action.

The copy that is commonly used in the serial ask has been in place since it was first launched. There have been a handful of attempts to test this language but without any valid results.

With the high motivation of year end giving, we knew that we would be getting significantly more donors than normal. Given this increased volume, we wanted to try a new value proposition that aligned better with the current political climate.



View the experiment »



-40.5% drop How providing a midpoint call-to-action impacts overall conversion

Date Added: January 3, 2018
Research Partner: The Heritage Foundation
Element tested: Advertising

For The Daily Signal, their Dear Reader inline donation ask had become a significant source of revenue for the organization. During the last calendar year end campaign, we did some testing around both the messaging and ways to increase urgency. Over the last 12 months, we had also learned that moving the ask to the bottom of the article was able to significantly increase donations. However, when we moved it to the bottom, we left a small, midpoint call-to-action that was a short headline and sentence that linked to the full offer at the bottom of the article.

During this year end campaign, we wanted to see if we could extrapolate what we learned from the positioning experiment. We wanted to remove the midpoint CTA entirely which would give readers an uninterrupted reading experience before the ask was made. Our hypothesis was that this would lead to an increased likelihood of someone engaging with the ad at the end.



View the experiment »


56.7% lift How matching gift copy affects clickthrough rate on a modal popup

Date Added: January 2, 2018
Research Partner: FamilyLife
Element tested: Advertising

FamilyLife was in the final week of their year-end campaign, and launched a modal popup to point web traffic to the donation page. The popup contained a countdown clock as well as urgency-focused message. The default version of the popup had language that echoed the headline, but they wondered if including matching gift language would increase clickthrough rate.

They created a treatment that kept urgent language, but added in the fact that the gift would be matched dollar-for-dollar. They launched a test to determine a winner.



View the experiment »


Not Valid How a radical redesign for a year-end donation page impacts conversion

Date Added: January 2, 2018
Research Partner: Focus on the Family
Element tested: Donation Page Design, Donation Page Headline, Donation Page Copy, Donation Page Form

During the Focus on the Family year-end campaign, we hypothesized that we might be able increase conversion by testing their current year-end donation page with a radical redesign. The radical redesign included removing the video on the page, adding a countdown clock on the page, increasing value proposition and added a giving thermometer.



View the experiment »