How the membership offer increased new donor conversion rates from Facebook Experiment ID: #17998

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Ended On: 8/28/2019

As a part of experimenting with ADF’s membership offer, we wanted to test a direct donation appeal from Facebook audiences being driven directly into the membership offer page. The experiment had a fixed budget and the goal was to determine the viability of the membership offer in terms of acquiring new donors. 

We decided to compare our fixed budget (once spent) and the results it generated against the “first x dollars spent” (that matched the budget) in two similar (and recently launched) offers to see if we could glean insight into how membership performed as a new donor acquisition offer.

Research Question

How does the Membership offer compare to our “best practice” approach to prospecting for new donors in Facebook?


C: The Larsen's Story
T1: Ruben's Story
T2: Membership Offer


Treatment Name Conv. Rate Relative Difference Confidence
C: The Larsen's Story 0.16%
T1: Ruben's Story 0.61% 275.8% 88.8%
T2: Membership Offer 1.9% 1,083.9% 100.0%

This experiment has a required sample size of 205 in order to be valid. Since the experiment had a total sample size of 4,769, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 1,083.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 99.9% level of confidence, we were achieved a +1084% increase in new donor conversion rate over “The Larsen’s Story” offer and a +215% increase in new donor conversion rate (albeit only on a 44% LoC) when compared to the “Ruben’s Story” offer.

Other observations we made about the Membership offer compared to the other two offers are:

  • We decreased the Gross Cost per New Donor (GCPND) value by: -81% compared to The Larsen’s and -70% when compared to Ruben’s Story.
  • We decreased the Net Cost per New Donor (NCPND) value by: -85% compared to The Larsen’s and -73% when compared to Ruben’s Story.
  • The number of NET-NEW donors acquired through the Membership offer was 220% higher compared to both of the other offers.
  • The average gift decreased by -18% when compared to The Larsen’s offer and -40% when compared to Ruben’s Story.
  • The ROI on new donor prospecting was increased by +343% when compared to The Larsen’s offer and 98% when compared to Ruben’s Story.
  • The cost per click (CPC) value for Membership was higher (+117%) when compared to The Larsen’s, but was about equal (-5%) when compared to Ruben’s Story.

This is further evidence that The Membership offer seems to have resonance with prospective new donors and that a direct membership appeal in Facebook can be as effective overall as one of our best performing offers (Larsen’s & Ruben’s stories).

Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.