191.2% lift How optimizing for a purchase in Facebook affects donor conversion

Date Added: January 30, 2019 Research Partner: Dallas Theological Seminary Element tested: Advertising

DTS was running a campaign on Facebook for their free Revelation online course. They typically use custom conversions in Facebook to track course signups, and Facebook shows ads to people in the targeted audiences based on this conversion. They recently discovered that Facebook has a native Purchase event that would allow them to optimize for donors, rather than course signups. This test involves a duplicate Revelation campaign where the only element changed is the conversion event- Custom Conversion vs. Purchase. 

View the experiment »

32.1% lift How an additional, clarifying form field affects webinar registrations

Date Added: January 30, 2019 Research Partner: NextAfter Element tested: Name Acquisition Form

In this experiment, we wanted to see if adding an additional field to understand whether or not someone worked for a nonprofit would make a significant impact on conversion. If we could add this field without hurting registrations, we'd be able to have cleaner data and do additional segmentation in our marketing to try to engage "for-profit" people on our email file differently. So we set up a 3 variant experiment - the control had our normal form fields, variant 1 had an additional required field asking if someone worked for a nonprofit, and variant 2 had an un-required field asking the same question.

View the experiment »

-26.1% drop How streamlining a mobile child sponsorship experience affects donor conversion

Date Added: January 30, 2019 Research Partner: Compassion International Element tested: Advertising, Donation Page Copy

Compassion International was examining its child sponsorship funnel and noticed that the mobile experience did not bring children who could be sponsored into the viewport. They wondered if this kept people from realizing that the page allowed them to sponsor children by forcing them to scroll. They developed a treatment that streamlined the "trust seals" into a horizontal row and removed the description of what happens during sponsorship. They launched an A/B test just on mobile to determine the impact.

View the experiment »

25.1% lift How a donor-based appeal (instead of a goal-based appeal) affects conversion rate

Date Added: January 30, 2019 Research Partner: CaringBridge Element tested: Advertising

CaringBridge had a matching gift offer as part of their year-end campaign. They had recently developed a dynamic thermometer that would update as a goal was reached and had leveraged it to increase conversion. However, they wondered if the large goal amount was too vague, and possibly caused too much cognitive friction in the mind of the reader. Did it make the reader wonder if the large goal could be achieved? Did it make the reader do mental math to approximate how much had been raised? Most importantly—did it communicate to the reader that their gift could have an impact?

They decided to run an experiment with a donor-based appeal, breaking their day down into hourly goals and keeping a dynamic countdown of how many donors were needed to meet the hourly goal. Their hypothesis was that smaller goals would bring it more within reach of the donor, and that communicating a goal in terms of how many donations—rather than how much money—was needed would tell the donor how they could materially affect the goal. Notably, the treatment did not include the match language at all. 

They launched an A/B test to understand the impact of this treatment. 

View the experiment »

149.8% lift How a change in sender influenced donations

Date Added: January 26, 2019 Research Partner: Buckner International Element tested: Email Sender

Buckner International had historically seen success introducing a new sender in their solicitation emails. Their standard sender, Dr. Albert Reyes, is the President and CEO of the organization and his name is used for the majority of cultivation and solicitation emails. However, we had never directly tested the impact of a different sender as a head-to-head experiment. For the January solicitation, we decided to do that.

We split the file in half for both of the solicitation emails sent that month. We made sure to maintain the split so that we could measure the impact of the new sender on more than just the first email.

View the experiment »

-20.5% drop How additional space in an email template affects unsubscribe rate

Date Added: January 22, 2019 Research Partner: EWTN Element tested: Email Copy

EWTN was running their year-end campaign while building an email preferences center in their new email platform. A discussion about unsubscribes led to their team noticing that the email footer, which contained the unsubscribe link, was really close to the bottom of the message—possibly too close. They were curious to see if simply adding in more whitespace led to a reduced unsubscribe rate. They created a treatment that simply introduced 10 lines of whitespace to push the footer down, and split the file to see how it affected unsubscribes.

View the experiment »

34.8% lift How we are able to use subject lines to learn an audience’s affinities

Date Added: January 20, 2019 Research Partner: Buckner International Element tested: Email Subject Line

Buckner International was sending out their monthly cultivation email to their house file. In recent months, one of the primary discussion points related to their subscribers revolved around how we could increase their overall engagement. The open rate of the file had slipped in recent months. In addition to evaluating technical solutions to the problem, we also wanted to see if we could identify topics that may reactivate people. Since many of these subscribers came from an offer related to scripture, we wanted to see if incorporating scripture earlier in the email (like the subject line) would increase their engagement.

View the experiment »

38.2% lift Does an email sent from an individual produce a higher open rate than an email sent from an organization?

Date Added: January 16, 2019 Research Partner: CRISTA Ministries Element tested: Email Sender

Traditionally during the Global Gift Guide campaign, emails are sent from World Concern. In an effort to increase the volume of people who open the emails being sent, we hypothesized that an email sent from a person rather than the organization might result in higher open rates. To test this, we developed a treatment email that was sent from Jacinta Tegman. The sender was the only variable that was different for the email; all other email content remained consistent with the control.

View the experiment »

35.0% lift How simplifying an email template improved open rates.

Date Added: January 16, 2019 Research Partner: CRISTA Ministries Element tested: Email Design

In reviewing historical promotional emails for the Global Gift Guide, it was clear that the organization has always leaned upon well designed and graphic-heavy images to promote the gift catalog. Our research on the subject has typically yielded improved results when stripping email templates down to "plain text" style emails (still with complete tracking and HTML-based) to appear as if they were written from an individual. We wanted to experiment with emails in 2018 to verify if this approach would also work for World Concern.

View the experiment »

18.4% lift How a donation platform can impact donor conversion

Date Added: January 16, 2019 Research Partner: The Missionary Oblates of Mary Immaculate Element tested: Donation Page Form

The Missionary Oblates of Mary Immaculate transitioned their donation platform in an effort to more efficiently create and change donation pages and for a better user experience. To test the impact of the new platform against the old platform we tested them against each other. The content on the donation pages remained the same and consistent so that we could isolate the platform itself. 

View the experiment »