-6.1% drop How a value proposition built upon “what” instead of “why” impacts email acquisition

Date Added: September 7, 2017 Research Partner: Dallas Theological Seminary Element tested: Name Acquisition Headline, Name Acquisition Copy

Dallas Theological Seminary had recently launched their first online course, a study of the Gospel of John. The course was offered for free in exchange for a visitor's name, email, and address. With significant traffic being sent to the site, we wanted to find the best way to communicate the value to the visitors. Using the learnings from experiments run with similar organization, we tried applying two key principles to the copy on the page:

  • We emphasized what the visitor would get by subscribing instead of what they would have to do
  • We reduced the friction associated with the process by removing copy that we felt was unnecessary to the conversion.
We then tested these two pages against each other and monitored results.

View the experiment »


-37.1% drop How video ads affect email acquisition rate for a free course

Date Added: September 6, 2017 Research Partner: Dallas Theological Seminary Element tested: Advertising

DTS was promoting their new course, called Can You Trust the Bible?. They had a special video from their Academic Dean to promote it and wanted to test using this video to promote the course. Video ads had shown in some experiments to improve conversion for offers, although DTS had previously used static ads. They split audiences and tested each ad unit to see which resulted in a better cost-per-acquisition.  

View the experiment »


-32.6% drop How clarity in the call-to-action affects email acquisition

Date Added: September 5, 2017 Research Partner: Dallas Theological Seminary Element tested: Email Design, Email Copy, Email Call-to-Action, Name Acquisition Design

Dallas Theological Seminary launched a new course called "Can You Trust the Bible?" They had shot a video of their Academic Dean explaining the course, and wanted to see how it performed against a direct invitation from the President to join the course. So they created two entrance paths to the course. The first was their control—a direct ask, via email, from the President of DTS. The second was an invitation from the President with a call-to-action to "watch the video to learn more about the course". They hypothesized that the softer call-to-action would drive more traffic to the landing page, and the short video with a clear call-to-action would produce more overall conversions. They split their file and closely monitored the data to determine a winner.

View the experiment »


74.0% lift How using scripture in Facebook ad copy affects email acquisition rate

Date Added: August 21, 2017 Research Partner: Dallas Theological Seminary Element tested: Advertising

Dallas Theological Seminary found that the treatment containing the scripture verse produced a 74% increase in traffic to their course signup page. It’s clear that this addition helped users understand with more clarity why studying the book of Genesis is important as Christians. One possible conclusion is that the use of scripture shifted the authoritative voice away from Dallas Theological Seminary and on to the Bible.

This discovery has led Dallas Theological Seminary to re-launch other courses with a similar approach.



View the experiment »


8.9% lift How reducing form field friction and anxiety affects conversion rate

Date Added: January 3, 2017 Research Partner: Dallas Theological Seminary Element tested: Name Acquisition Form

Dallas Theological Seminary launched a new free course studying the book of Revelation. Traditionally, they had always asked for name, email address, and full home address in the registration process so they could send prospects direct mail if needed. However, the data had shown that these prospects did not respond well to direct mail unless they made an online gift during the course. Since DTS got all the home address data when they made a gift, they didn't need to collect it at the beginning. They were curious how this might affect course registrations, so they launched a test between a full signup form and one that just asked for name, email, and password. They split the traffic to determine the effect of removing the home address fields.

View the experiment »


22.5% lift How subject line personalization affects open rate

Date Added: November 23, 2016 Research Partner: Dallas Theological Seminary Element tested: Email Subject Line

Dallas Theological Seminary was promoting their annual Christmas devotional series to their house email file. In the past, they had used a very straightforward subject line to promote the offer: Devotionals for the Christmas Season. But their new marketing platform allowed them to personalize the subject line with the recipient's firstname, so they put it to a test. They sent 10% of their house file each treatment and collected data to determine a winner.

View the experiment »


55.1% lift How a premium offer affects donor motivation

Date Added: November 13, 2016 Research Partner: Dallas Theological Seminary Element tested: Donation Page Copy

Dallas Theological Seminary had released a free online course on the book of Revelation. After visitors registered for the course, they were given the opportunity to give a gift to support the free courses. DTS had traditionally used commentary books as premiums for donations, but wanted to test the impact of removing the premium offer on donations and revenue. They hypothesized that removing the book might result in more donations, though the average gift might be lower without the "anchor point" ($75, in this case) for the book offer. They launched an A/B test to determine a winner.

View the experiment »


130.8% lift How disclosing the amount raised affects conversion

Date Added: September 20, 2016 Research Partner: Dallas Theological Seminary Element tested: Email Copy

Dallas Theological Seminary was raising money in honor of North Texas Giving day, and had a group of donors pledge $50,000 if they could raise $50,000 in a week. They had written an email to existing donors that was very personal in tone, and wanted to test the impact of telling the donors how much had been raised up to that point. They created a treatment email that included a single extra line of copy: "As of right now, we've already raised half of that amount." They split the donor file and sent one treatment to each segment to see if they could quantify the impact of this single statement.  

View the experiment »


47.6% lift How congruence between ad and landing page affects conversion

Date Added: September 13, 2016 Research Partner: Dallas Theological Seminary Element tested: Advertising

Dallas Theological Seminary was promoting their second course, Genesis, on Facebook. They wanted to emphasize the academic nature of the course, so they launched ads that showed a person taking the course while studying. But they hypothesized that they might be able to increase conversion by employing the principle known as congruency: aligning each element of a marketing funnel to increase response. They pulled the image of the Sistine Chapel from the landing page and put it into the ad, and launched an A/B test to see if they were able to affect conversion.

View the experiment »


Not Valid How registration form length affects email acquisition rate

Date Added: June 27, 2016 Research Partner: Dallas Theological Seminary Element tested: Donation Page Form

Dallas Theological Seminary had just released their third free online course on Genesis. Although they had made plenty of changes to the landing page copy, they hadn't tried to optimize the form, which asked for both online and offline information, totaling 11 fields. We hypothesized that reducing the perceived length of the form (by nesting fields together) would increase conversions.

View the experiment »