Not Valid How removing a video affects email appeal revenue

Date Added: December 14, 2017
Research Partner: Focus on the Family
Element tested: Email Design, Donation Page Design

The team at Focus on the Family had seen experiments from other NextAfter partners suggesting that video on donation forms could actually be harming conversion.  With an appeal featuring video set to go out, they decided to test into this effect to see whether their audience would respond the same.  This particular test was against a segment of existing donors, although a non-donor segment was also tested.



View the experiment »


276.7% lift How elaborating on specific organizational initiatives affects clickthrough rate

Date Added: December 13, 2017
Research Partner: Dallas Theological Seminary
Element tested: Email Copy

Dallas Theological Seminary recently began their calendar year end campaign, focused three pillars: The Call to Teach, The Will to Engage, and The Heart to Go. As they prepared to send an appeal on email focused on the second pillar–The Will to Engage– they wondered if an email that elaborated on the strategic initiatives they’ve created to engage culture would increase the clickthrough rate to the donation page. They created a treatment version and launched an A/B test to find out.



View the experiment »


-5.3% drop How inviting an email recipient to participate affects email open rate

Date Added: December 13, 2017
Research Partner: Hillsdale College
Element tested: Email Subject Line

Hillsdale College recently began their calendar year end campaign. This year they decided to focus more specifically on their independence and how it enables them to fulfill their mission–without accepting EVEN ONE PENNY of government funding! One of their appeal emails focused on their presence in Washington D.C. through the Kirby Center. Before sending, they wondered: Will recipients be more motivated to open an email that invites them to participate in an action (in this case, helping teach policymakers about the constitution). They created an A/B subject line test and monitored the results.



View the experiment »


Not Valid How a Facebook headline affects direct-ask donor conversion

Date Added: December 12, 2017
Research Partner: The Heritage Foundation
Element tested: Advertising

The Heritage Foundation noticed an opportunity on Facebook to retarget visitors who viewed their online donation pages. They created to direct ask Facebook ads that invited anyone who visited the page but did not complete the form to return and support the Heritage Foundation. In the first ad, they simply asked all visitors (including members who visited the page) to ‘Donate now.’ In the second ad, the asked all non-member visitors to join the Heritage Foundation by ‘Becoming a member today.’ They launched a test to find out which approach would be more effective at converting donors.



View the experiment »


Not Valid How supporting copy length on a Facebook ad affects email acquisition rate

Date Added: December 11, 2017
Research Partner: Alliance Defending Freedom
Element tested: Advertising

Oral arguments for Jack Phillips–a client of Alliance Defending Freedom–began on December 5th. As the news covered this national story, Alliance Defending Freedom decided to launch a Facebook ad campaign urging people to sign a letter of support for Jack. Since they knew some people were unfamiliar with Jack, they wondered: Will increasing the length of supporting ad copy on a facebook ad increase clickthrough rate? To test this, they created a longer treatment and launched an A/B test to find out.



View the experiment »


Not Valid How a direct ask via voicemail impacts donor giving behaviors

Date Added: December 8, 2017
Research Partner: Buckner International
Element tested: Advertising

For Giving Tuesday, Buckner International wanted to test out a new channel of communication to see if we could boost donor giving. We utilized a service named SlyBroadcast that allows us to make straight to voicemail phone calls. It is a way of making a phone call to thousands of participants that would immediately show up as a missed phone call on their phone. While it is a recorded message, it would appear to be a genuine voicemail given the nature of the technology.

We had previously used SlyBroadcast for cultivation and stewardship messages to donors. This time around, we wanted to make a direct ask and request that donors go online or call back to make their Giving Tuesday gift.

We focused our audience on those that had both a phone number and email address on file. We then split this audience in half by their highest previous contribution levels to create equal segments so that we could create a treatment and control group that would not be skewed by past giving.



View the experiment »


Not Valid How the total number of questions in a quiz affects email acquisition

Date Added: December 8, 2017
Research Partner: Illinois Policy Institute
Element tested: Name Acquisition Design

This year, Illinois Policy Institute has exposed a serious problem in the state of Illinois. It’s been more than 15 years since the Illinois General Assembly has passed a balanced budget, and lawmakers have turned to tax hikes in order to ease the deficit. However, this is not a long-term solution and it’s caused illinoisans to pay some of the highest taxes in the nation. Illinois Policy developed a plan to balance the budget and ease taxes, all at once. In order to get the word out, they developed a 9-question quiz followed by a tax petition that would build awareness among Illinoisans and encourage them to take action to advance this plan. Before launching, Illinois Policy Institute wondered: Will a shorter, 6-question quiz increase petition signatures?

They created a treatment version and launched an A/B test to find out.



View the experiment »


Not Valid How adding region-specific wording to an email affects click rate

Date Added: December 5, 2017
Research Partner: Buckner International
Element tested: Email Copy

Buckner International was giving a free offer to people in their email file as a gift for Giving Tuesday. They wanted to test whether adding region-specific copy in the email would increase clickthrough rate. The control had generic language, whereas the treatment would use contact-specific wording and apply their most local Buckner branch’s name in that copy.



View the experiment »


13.3% lift How the subject line of an email affects open rate

Date Added: December 5, 2017
Research Partner: Buckner International
Element tested: Email Subject Line

Buckner International sent out an email before Giving Tuesday that let readers know to watch their email for a gift they would be sending them tomorrow. We decided to do a subject line test to see if we could learn more about what motivated Buckner subscribers. For this campaign, we wanted to test whether a mysterious subject line would perform better than one that indicated something was going to be given to them.



View the experiment »


Not Valid How a banner header image impacts article engagement for desktop users

Date Added: December 5, 2017
Research Partner: Alliance Defending Freedom
Element tested:

Alliance Defending Freedom was interested in knowing how blog page design impacted reader engagement on their site. They wondered: Will removing the header image on blog pages decrease friction and increase reader engagement? They created a trackable goal that measured clicks from one blog to a second article. They then created a treatment version that removed the header image and launched an A/B test to find out.



View the experiment »