How optimizing for a donation vs. a purchase in Facebook affects donor conversion Experiment ID: #11751

Dallas Theological Seminary

The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.

Experiment Summary

Timeframe: 1/30/2019 - 2/12/2019

DTS previously validated an experiment that increased donor conversion for the Revelation online course by optimizing for the native Facebook event “Purchase” rather than a custom conversion, or course signup. In test #10578, optimizing for purchase targeting led to a 191% increase in donors from the Revelation campaign, as well as a 31% increase in average gift from the “purchase” campaign. This confirmed our hypothesis that Facebook shows ads to more donor-motivated people when optimizing for purchases. It allows them to pull in additional purchase data from Facebook instead of relying primarily on data that we give to them (for example, a list of prospects or a lookalike audience). 

In this test, we utilized the native event on Facebook labeled “Donate”, to test if the same approach would increase donations even further knowing that Facebook was likely pulling in nonprofit or donor data as well as purchase data. 

All other campaign elements remained the same. 

Research Question

Does optimizing for a donation vs. a purchase increase donor conversion? 

Design

C: Purchase Optimization
T1: Donate Optimization

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Purchase Optimization 0.12%
T1: Donate Optimization 0.07% -42.9% 97.9%

This experiment has a required sample size of 26,919 in order to be valid. Since the experiment had a total sample size of 74,558, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 42.9% decrease in conversion rate
× 0% increase in average gift

Key Learnings

Optimizing for a donation decreased conversion rate by 41%, which further validates our learning that the purchase event has the highest likelihood of converting an instant donor on Facebook. Further testing is needed to determine what metrics Facebook is using to determine ideal donors and purchases. 


Experiment Documented by...

Allison Jones

Allison is an Optimization Associate at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.