129.9% lift How images in an email affect clickthrough rate

Date Added: May 3, 2016 Research Partner: Dallas Theological Seminary Element tested: Email Design, Email Copy

Dallas Theological Seminary was promoting their newest online course on Genesis to their house file. Many of the people in their house file had already taken one of their first two courses. Previous testing had shown that their prospects responded well to very personal invitations. However, they wanted to try a more graphical approach that presented the new course in a more dramatic way and gave the reader more to click on. They created a treatment that inserted an image from the course into the copy (linked to the registration page) and put an image of Dr. Mark Bailey, the sender, next to his signature. They split the file and sent an A/B email test to determine a winner.

View the experiment »


34.7% lift How adding a gift array affects conversion

Date Added: March 3, 2016 Research Partner: Dallas Theological Seminary Element tested: Donation Page Form

Dallas Theological Seminary launched a free online course to grow their email file and attract new donors. After a visitor signed up for the course, they were presented with an opportunity to give, accompanied by a premium offer to receive a Bible commentary. Previous testing and optimization with DTS had shown that their donors preferred an open field instead of a gift array with recommended gift amounts. But as this course began to take off, they realized they were bringing new people to the page. They hypothesized that these new people might benefit from a choice of gift options rather than a blank field. So they created a second page with the same copy, but three gift options above the donation form and ran an A/B split test to find out which one resulted in more conversions and donation revenue.

View the experiment »


Not Valid How value proposition affects conversion rate

Date Added: February 19, 2016 Research Partner: Dallas Theological Seminary Element tested: Name Acquisition Headline

Dallas Theological Seminary had a great response to their first free online course on the Gospel of John. But objectively looking at the headline 0f their signup page a few weeks later, they wondered if the value proposition could be improved -- specifically, by using something other than the word "study". Other experiments had shown that offering the benefits of the offer often outperform words like "study" that make the offer seen like work. So they launched two headline tests to see if they could affect conversion. Both headlines sought to show the real opportunity of the course -- to "step inside the classroom", but with different desired effects. The first headline offered the opportunity to "see the gospel of John like never before" and the other to "grow your understanding of John's gospel". They launched an A/B/C test to determine a winner.

View the experiment »


-18.1% drop How removing creative elements affects conversion

Date Added: February 10, 2016 Research Partner: Dallas Theological Seminary Element tested: Name Acquisition Design

Dallas Theological Seminary launched a free online course studying the Gospel of John that was enrolling students at a rapid clip. DTS wanted to see if removing creative elements would improve the clarity of the offer by allowing the eye to focus on the headline, which delivered the value proposition. They also wanted to test a different headline that used another word than "Study", which conveyed work to the visitor. So they launched an experiment to determine whether streamlining the page would improve clarity and increase visitor-to-student conversion.

View the experiment »


25.7% lift How “activation” language affects clickthrough rate

Date Added: December 8, 2015 Research Partner: Dallas Theological Seminary Element tested: Advertising

Dallas Theological Seminary was promoting a free online course taught by the president of the Seminary. They launched an ad campaign on Facebook to generate interest in the course, and saw a tremendous response. This led them to think that their prospect targets might be much more qualified and ready to sign up. So they created a second ad that used "Activate your free course now" as a call-to-action and A/B tested them to see which attracted more clicks.

View the experiment »


553.3% lift How refining the donation value proposition affects conversion

Date Added: December 2, 2015 Research Partner: Dallas Theological Seminary Element tested: Donation Page Design, Donation Page Copy

Dallas Theological Seminary launched a resource center for biblical perspectives on today’s issues. The goal of this resource center was email and donor acquisition. Immediately after the signup page was a donation ask. DTS had run a similar value proposition experiment on this page before but had seen no significant difference in results. Taking those learnings, they did a more radical test. They tested which value proposition was stronger: an ask for an investment in a student or an ask supporting the provision of the resource center. They also changed the placement of the endorsement and added a photo of the endorser.

View the experiment »


19.8% lift How additional ad copy affects clickthrough rate

Date Added: November 30, 2015 Research Partner: Dallas Theological Seminary Element tested: Advertising

Dallas Theological Seminary was promoting their Christmas devotional series as a free offer on Facebook. Previous testing had shown that using copy in the image of the ad increased clickthrough rate and conversion rate. But as they reviewed their ad, they wondered if there was actually too much copy in the ad, and reading it bogged the reader down and kept them from clicking. They made a treatment that removed the subheadline from their ad image, leaving only a single call-to-action with no supporting copy in order to simplify the value proposition. They launched an A/B test to determine a winner.

View the experiment »


37.3% lift How subject line copy that “teases” affects open rate

Date Added: November 20, 2015 Research Partner: Dallas Theological Seminary Element tested: Email Subject Line

Dallas Theological Seminary launched a free online course studying the Gospel of John. They were promoting it by renting email lists, and wanted to determine which subject lines encouraged more recipients to open the email. They developed two subject lines based on hypotheses about the prospect. The first was very direct: "Study the Gospel of John with DTS". The second teased the offer, and expanded from the acronym: "A special offer from Dallas Theological Seminary".

View the experiment »


Not Valid How the wrong value proposition can affect conversion

Date Added: August 31, 2015 Research Partner: Dallas Theological Seminary Element tested: Donation Page Headline, Donation Page Copy, Donation Page Form

The primary donation form for Dallas Theological Seminary (DTS) had remained consistent for the last several years. It was a well-designed form but it made the assumption that those donors that make it to the page are already convinced to give their gift. This was evident from the lack of any copy or significant calls to action at the top of the form. In an effort to increase the perceived value of the donation to potential donors, we decided to create a new treatment. This new page would utilize the same donation form but would add copy to the top that discussed one of the unique value propositions of DTS. It also included quotes from well known figures in the Christian community supporting the work of DTS.

View the experiment »


Not Valid How reducing copy for highly motivated donors affects conversion

Date Added: August 14, 2015 Research Partner: Dallas Theological Seminary Element tested: Donation Page Copy

Dallas Theological Seminary sent a survey to their donors and prospects to better understand what they valued most about the organization. The survey was immediately followed by a donation ask. DTS hypothesized that after completing the survey, donors wouldn't need as much copy to be convinced to make a gift. So they developed a treatment landing page that removed much of the copy -- which to a highly motivated donor, could pose as friction.

View the experiment »