Not Valid How leading with a question impact email click rate

Date Added: July 23, 2019 Research Partner: Alliance Defending Freedom Element tested: Email Copy

Alliance Defending Freedom was sending out one of their weekly cultivation emails as part of a longitudinal experiment to improve email engagement. In the previous week, we had observed a surprising result in an experiment. When starting the email with a personal anecdote, we saw a decreased level of engagement. We had a hypothesis that this may be due to the way the two emails differed in their introductions. The control started with the question to the reader while the treatment started with the story. Since there were several other factors that could have led to the observed result, we decided to isolate the question factor and run another A/B split. The only difference in the control and treatments email would then be whether they begin with a question or a statement.

View the experiment »


13.1% lift How sticky bars affect conversion in non-match periods

Date Added: July 22, 2019 Research Partner: CaringBridge Element tested: Donation Page Copy

CaringBridge had run sticky bars on their donation pages to promote matching grants during their campaign months. While this tactic had generally lifted response by reinforcing the match, they wanted to see if it could be effective in non-match times as well. They created two different treatments for the sticky bar—one that reminded them that CaringBridge is donor-supported, and one that thanked them for their generosity and included a photo and name of a CaringBridge staff member, for increased personalization.

View the experiment »


-36.2% drop How adding a personal story impacted email engagement

Date Added: July 22, 2019 Research Partner: Alliance Defending Freedom Element tested: Email Copy

Alliance Defending Freedom was continuing the longitudinal experiment to increase cultivation to a targeted segment of their email house file. The goal of these emails was to simply engage the audience and provide them with ongoing content and stewardship. With that in mind, we wanted to find ways to increase that engagement. One hypothesis we developed was that we could potentially increase the personal-feel of these emails by including stories from the sender's own life. We believed this personal touch might increase the likelihood that someone would click on the email.

View the experiment »


246.7% lift How adding social proof impacts email sign ups

Date Added: July 22, 2019 Research Partner: NextAfter Element tested: Name Acquisition Copy

When we redesigned the NextAfter blog, one of the goals was to increase subscribers/email signups through both the placement and message of the offer. One new sign up placement was a  a large call to action area above the blog and we wanted to see which message would encourage people to sign up. For the control, we kept our old/standard value proposition about what you will get if you sign up but for the treatment, we added in the number of people subscribed and who they were — fundraisers and nonprofit marketers — to add social proof and clarity.

View the experiment »



-9.0% drop How a CPA bid strategy decreased ROI

Date Added: July 12, 2019 Research Partner: The Heritage Foundation Element tested: Advertising

Within the Google Ads platform, we decided to test a CPA bid strategy to see if we could increase our level of competitiveness and generate more clicks and donors more effectively than with a typical impression-based bid strategy.

View the experiment »


93.3% lift How presenting an email acquisition opportunity as a forced pop-up impacts conversion

Date Added: July 12, 2019 Research Partner: Billy Graham Evangelistic Association Element tested: Name Acquisition Form

Billy Graham Evangelical Association gets significant traffic to their Answers pages. At the bottom of these pages there is an opportunity for people to sign up to receive the BGEA newsletter. We hypothesized that presenting that acquisition opportunity in a different way might increase conversion and name acquisition for the organization. To test this, we developed a pop-up/exit-intent pop-up that appeared on desktop (and a mobile slide-up). Knowing that a lot of the people coming to the Answers pages were returning visitors, and that a forced pop-up on a page can cause a poor user experience, we delayed the pop-up from appearing until after a person had been on the page for 30 seconds. If they exited prior to that time and seeing the pop-up, we would then trigger an exit-intent pop-up. In addition to this, we also put a rule on the pop-ups to only show every third visit.

View the experiment »


24.7% lift How adding value proposition copy on a homepage can impact member conversion

Date Added: July 12, 2019 Research Partner: Focus on the Family Element tested: Name Acquisition Copy

On the homepage of the Adventures in Odyssey website, people have the opportunity to become a member (sign up for the Adventures in Odyssey club). When we looked at the page, we hypothesized that there wasn't clarity in what people were signing up for, or a clear call-to-action. The homepage communicated what the benefits/incentives of being a member were, but it didn't tell a visitor coming to the website why a person should become a member. We developed a treatment that added value proposition to the homepage prior to explaining the benefits of the membership. We also added a call-to-action at the bottom of the benefits section in an effort to increase member sign-ups from the homepage. 



View the experiment »


64.3% lift How location based imagery affects click-through rate of a Facebook Ad campaign

Date Added: July 12, 2019 Research Partner: NextAfter Element tested: Advertising

In this experiment, we were advertising online fundraising workshops hosted by NextAfter Institute. Each specific location had it's own geo-targeted Facebook Ad campaign. The intention was to keep the workshop in their mind during the course of the sales period and drive them back to the landing page to consider attending. We knew our sample size would be too small to optimized for ticket sales, but we hoped to run an experiment to optimize our landing page click-throughs. The control ads used a picture of a course instructor teaching in a room full of attendees. Common advertising recommendations are to use people in imagery to make it relatable. But we wondered if using images specific to someone's city would actually be more a powerful driver to show that the training was more specific and local to them.

View the experiment »


38.0% lift How the value proposition of a donation page affects donor conversion

Date Added: July 12, 2019 Research Partner: Dallas Theological Seminary Element tested: Name Acquisition Copy

DTS was running an experiment on their Acts course donation page that centered around the value proposition of the page. This copy treatment has been tested and validated on the Genesis course, but the last time it was tested, it was tested with the additional removal of the premium from the page. In order to isolate the value proposition copy variable, we've retested this with the premium on both the control and treatment, with the only change being the body copy.

View the experiment »