The impact of a standard Facebook event vs a custom event on volunteer signups - NextAfter
Care Net

The impact of a standard Facebook event vs a custom event on volunteer signups

Experiment ID: #66389

Care Net

Experiment Summary

Ended On: 07/13/2021

For more than a year, we’ve been running a campaign on Facebook to acquire volunteers for our Making Life Disciples initiative. We had an ad that pointed to a pledge signup page. Those who signed the pledge would then land on a volunteer signup page. We had been using a custom conversion that targeted the volunteer confirmation page. We knew from our donation experiments that using the standard “purchase” event in Facebook increased conversions when tested against a custom conversion targeting our donation pages. We wondered if a similar principle would work with testing Facebook’s standard “registration” conversion event against our custom conversion.

Research Question

We believe that using Facebook’s registration conversion event for our volunteer campaign will achieve greater conversion rates and lower cost-per-signup.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 2.7%
T1: Treatment #1 7.0% 154.7% 100.0%

This experiment has a required sample size of 211 in order to be valid. Since the experiment had a total sample size of 32,500, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 154.7% increase in conversion rate
× 0% increase in average gift

Key Learnings

When looking at email acquisition, we increased signatures on the pledge (step 1) by 155% by using the standard event. Now, signatures that came through the custom event converted to signups at about a 30% increased rate but we only had 60% validity on that metric. Since we were getting so many more users through step 1 using the standard conversion, even at a lower conversion rate on step 2, we still converted more pledge signers into volunteers. We also reduced our cost per signup from $37 to $15 by using this standard event.

It is possible that Facebook’s algorithm was unable to collect enough conversions on the custom event to properly optimize for it. But either way, the standard event was clearly better and led to more emails acquired and a lower volunteer signup conversion cost.


Experiment Documented by NextAfter

Question about experiment #66389

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.