98.0% lift How the style of offer impact email acquisition

Date Added: March 27, 2019 Research Partner: Alliance Defending Freedom Element tested: Name Acquisition Copy

In a previous experiment for Alliance Defending Freedom, we had found that a Statement of Belief dramatically outperformed the standard email newsletter offer for visitors coming from paid social media ads. Based upon these results, we wanted to see if the same impact would be seen if we applied the Statement of Belief offer to organic visitors to the blog. The ADF blog is a significant source of visitors to the site and, over the years, has become a great source for email acquisition.

View the experiment »

297.6% lift How optimizing for a purchase in a Facebook campaign affects instant donor conversion

Date Added: March 27, 2019 Research Partner: Dallas Theological Seminary Element tested: Advertising

DTS was running a campaign on Facebook for their free Hebrews online course. They typically use custom conversions in Facebook to track course signups, and Facebook shows ads to people in the targeted audiences based on this conversion. They recently discovered that Facebook has a native Purchase event that would allow them to optimize for donors, rather than course signups. This test involves a duplicate Hebrews campaign where the only element changed is the conversion event- Custom Conversion vs. Purchase. 

View the experiment »

Not Valid How email design affects click-through in a digest or newsletter email

Date Added: March 18, 2019 Research Partner: NextAfter Element tested: Email Design

Email appeal experiments had given us a predisposition to think that email with button, images, and minor design elements are ineffective. But we had really only discovered that to be true in the specific context of a donation appeal.

In this experiment, we wondered if basic design elements could actually lead to greater click-throughs. The control was a plain-text style email in the format we had tested into, and the treatment added section images and clickable html buttons.

View the experiment »

Not Valid The impact of Scripture in an appeal

Date Added: March 18, 2019 Research Partner: Buckner International Element tested: Email Copy

Buckner International had seen success with recent communications by leaning heavily into scripture. As a Christian organization, they would incorporate scripture verses into their cultivation emails and even had an acquisition offer that focused on scripture. With this in mind, we had the hypothesis that using scriptures in our appeals may increase the reader's sense of duty which would then increase the likelihood to respond. However, we wanted to be sure it wasn't isolated to a single email. To do this, we split the house file equally among donors and nondonors and maintained this split through the whole campaign. We then three distinct emails, each with a scripture and non-scripture, version to measure the impact on the gifts.

View the experiment »

66.9% lift How allowing Facebook to choose our audience improved donor conversion rate

Date Added: March 13, 2019 Research Partner: Americans for Prosperity Element tested: Advertising

Americans for Prosperity was interested in diversifying their audience more instead of only attracting "right of center" audiences through pre-programmed target lists. We decided to run a test to see if allowing Facebook's algorithm to choose who to run our ads to would impact our ability to convert donors.

View the experiment »

Not Valid How emphasizing matching gifts during the serial ask impacted revenue and donor conversion.

Date Added: March 13, 2019 Research Partner: Heritage Action for America Element tested: Donation Page Copy

During the calendar year end campaign for Heritage, we installed an exit popup which used social proof language along with a symbolic gift ask to people looking to leave the page (on desktop computers only) to try to increase donor conversion rate and revenue increases. We wanted to see if we could increase revenue while maintaining the donor conversion rate when using language to reinforce matching your gift amount to Heritage Action to the same amount the donor just selected to give to the Heritage Foundation.

View the experiment »

81.6% lift How links are placed in a digest email affects clickthrough

Date Added: March 13, 2019 Research Partner: NextAfter Element tested: Email Call-to-Action

In this experiment, we were continuing to test the format of our digest emails to optimize for more clicks. When setting up this email, we had 2 ideas of how we could link up the various articles, blogs, podcasts, etc. On the one hand, we could link up the description text so that the links are tied directly to the value copy. On the other hand, we could simply list out several links in a block below the description text.

View the experiment »

-37.6% drop How a different value proposition affected enrollment in the benchmark report

Date Added: March 13, 2019 Research Partner: NextAfter Element tested: Name Acquisition Copy

In this experiment, we wondered if our copy wasn't communicating enough about why the benchmark report is valuable. The control seemed to place a stronger emphasis on what is required to enroll in the report. So we created a treatment that was more focused on the value of a benchmark report to the marketer/fundraiser.

View the experiment »

-19.5% drop The impact of reinforcing the security of the email address field

Date Added: March 13, 2019 Research Partner: Alliance Defending Freedom Element tested: Name Acquisition Design

Alliance Defending Freedom had been running a statement of belief acquisition offer on their blog for well over a year. In experiments with similar nonprofits, we had discovered that adding visual security indicators to the credit card section of a donation form increased donor conversion. Our hypothesis was that this was due to isolating and reducing the mental friction associate with the area of the form that created the most anxiety. With this observation, we theorized that adding the same treatment to the email field of an acquisition form would yield similar results.

View the experiment »