Not Valid How reduced copy and a more direct headline affect donor conversion

Date Added: September 10, 2017
Research Partner: Dallas Theological Seminary
Element tested: Name Acquisition Headline, Name Acquisition Copy

Dallas Theological Seminary offered a free online course called How to Read the Bible Like a Seminary Professor to allow their donors and prospects to experience the value proposition of the Seminary. Although they had a good donor conversion rate, they wanted to see if they could improve email acquisition by making the value proposition on the signup page more appealing.

They created a treatment with a more direct headline: “Learn to read the Bible like never before”. Then, they removed the bulleted list from the copy to reduce the overall length of the copy, hypothesizing that it was not core to the value proposition on the page.

They split traffic to the page and launched an A/B test to determine a winner.



View the experiment »


Not Valid How removing styling from email affects donor conversion rate

Date Added: September 10, 2017
Research Partner: Hillsdale College
Element tested: Email Design

Hillsdale College was fundraising during their annual Constitution Day campaign. In previous emails, they had found that stripping out branding from their emails improved clickthrough rate and donor conversion rate. Their email template had been stripped to the bare essentials, yet still had a border around it with a separate background color. They decided to test this against a pure plain-text email that removed all styling entirely.

They split their file evenly and sent an A/B test to determine a winner.



View the experiment »


Not Valid How additional premium-focused value proposition affects conversion

Date Added: September 10, 2017
Research Partner: Dallas Theological Seminary
Element tested: Donation Page Design, Donation Page Copy

Dallas Theological Seminary had found a popular offer in their free Gospel of John course. After the user signed up, they were given the opportunity to make a gift.In exchange for a gift of $100 or more they were offered a commentary as the course companion book for the Gospel of John course. DTS decided to try to see if they could increase conversion rate by integrating review content for the commentary into the donation page. They pulled some reviews from a popular book review site onto the page, showed five stars, and gave a positive review for the commentary, hoping that this would increase conversion.



View the experiment »


Not Valid How a time-sensitive value proposition affects conversion rate for donors

Date Added: September 10, 2017
Research Partner: Dallas Theological Seminary
Element tested: Donation Page Copy

Dallas Theological Seminary was running a fundraising campaign for North Texas Giving Day.They had previously tested into a value proposition centered around their mission: growing the kingdom. However, they wanted to test to see if a very time-sensitive specific matching challenge ask would be a greater motivator for donors and nondonors. Since the two segments had different levels of affinity and motivation, they wanted to test them separately. They created a treatment page and showed it to both donors and nondonors independently, with a 50-50 split to try to determine which value proposition was a greater motivator for this specific campaign.



View the experiment »


Not Valid How a time-sensitive value proposition affects conversion rate for nondonors

Date Added: September 10, 2017
Research Partner: Dallas Theological Seminary
Element tested: Donation Page Copy

Dallas Theological Seminary was running a fundraising campaign for North Texas Giving Day.They had previously tested into a value proposition centered around their mission: growing the kingdom. However, they wanted to test to see if a very time-sensitive specific matching challenge ask would be a greater motivator for donors and nondonors. Since the two segments had different levels of affinity and motivation, they wanted to test them separately. They created a treatment page and showed it to both donors and nondonors independently, with a 50-50 split to try to determine which value proposition was a greater motivator for this specific campaign.



View the experiment »


393.4% lift How a more personalized value proposition affects remarketing conversion rate

Date Added: September 10, 2017
Research Partner: Hillsdale College
Element tested: Advertising

Hillsdale College decided to launch remarketing ads on Facebook to users who visited the signup page for their acclaimed “Constitution 101” course. They deliberated about the copy on these remarketing ads—should it re-state the original value proposition of the course, or should it encourage them to finish completion of the course?

Since this was a testable hypothesis, they created two ads. The first told the user that their course was still available, and encouraged them to finish signup. The second used a standard value proposition that encouraged them to grow their understanding of the Constitution through the course, but didn’t show any indicator that they had shown previous interest.



View the experiment »


5.2% lift How a video preview affects email acquisition rate

Date Added: September 10, 2017
Research Partner: Hillsdale College
Element tested: Name Acquisition Design

Hillsdale College was promoting a brand new course, Introduction to the Constitution. This course was shot in a manner that was very different from their previous courses, and had a much improved look and feel to the video. They realized that their original Introduction to the Constitution registration page expressed this improvement in video quality through text, but did not show the prospect the actual video of the course they were signing up for. They decided to test a video preview on the email acquisition page, even though they had previously tested video and founded it reduced conversion rate. They hypothesized that in this case, if the user was able to see the new course, they would be more likely to sign up. They created a treatment with small cosmetic adjustments to the page that included a video preview next to the sign up form.



View the experiment »


Not Valid How gift array amounts affect conversion rate and average gift

Date Added: September 10, 2017
Research Partner: Americans for Prosperity
Element tested: Donation Page Form

Americans for Prosperity was promoting a free eBook on Socialism, and asking for a donation on the following page. They were getting a decent conversion rate, but the average gift fell below their desired target. They noticed that the gift array on the page started at $5, and many donors gave donations below $25.

They had analyzed data  that showed that donors who gave $30 or more ended up being “higher quality” donors—meaning that they gave more over their “lifetime” and had higher retention rates.

So they developed an experiment to see if reducing the number of suggested gift amounts and starting the array at $25 would increase average gift without affecting overall revenue. They hypothesized that conversion rate might drop, but that the donors who came through would give more and therefore be “higher quality” donors.



View the experiment »


-26.4% drop How increasing the page length and complicating the message affected donor conversion

Date Added: September 10, 2017
Research Partner: Harvest Ministries
Element tested: Donation Page Copy

Harvest Ministries promoted their midmonth premium offer for the month which was, Thoughts to Make Your Heart Sing by Sally Lloyd-Jones. The email pushed people to the landing page to get the premium.

We hypothesized that if we added additional supportive organization-focused value proposition language we might be able to motivate more donors and increase conversion rate.

We created a treated and ran an A/B test to determine which would perform better.



View the experiment »


297.7% lift How an exit intent offer impacts email acquisition rate on desktop devices

Date Added: September 8, 2017
Research Partner: Alliance Defending Freedom
Element tested: Advertising, Name Acquisition Form

In a previous experiment, Alliance Defending Freedom discovered that exit intent offers dramatically increase email acquisition rate when shown to blog visitors. They wondered: Will there be a similar increase in email acquisition rate if an exit intent offer is shown to visitors on the ‘News/Press Release’ page? They set up a treatment using their winning exit intent offer and launched a test on desktop devices to find out.



View the experiment »