How purchase optimization affects donor conversion in a Facebook campaign Experiment ID: #12109

Hoover Institution

Experiment Summary

Timeframe: 1/29/2019 - 2/21/2019

Hoover Institution was running a Facebook campaign to promote their Intellectual Poverty offer, and wanted to test the effectiveness of optimizing for Facebook’s native Purchase event, which brings in outside financial data to inform targeting, on instant donor conversion. The control version of the campaign optimized for a custom conversion (ebook signup). Both campaigns were identical in ad set targeting and content.

Research Question

Does purchase optimization in a Facebook campaign increase instant donor conversion?


C: Custom Conversion Optimization
T1: Purchase Optimization


Treatment Name Conv. Rate Relative Difference Confidence
C: Custom Conversion Optimization 0.02%
T1: Purchase Optimization 0.05% 194.6% 98.6%

This experiment has a required sample size of 21,203 in order to be valid. Since the experiment had a total sample size of 113,472, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 194.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment campaign increased donor conversion rate by 194% over the course of about 20 days. This is consistent with what we’ve seen across other organization over the last few weeks, and encouraging as we push forward to increase donors significantly over the next few months.

The purchase optimization not only increased donations, it also produced a slight increase in signup conversion rate (3%).

We’ve turned off the original campaign and pushed spend to this campaign as a result.

Experiment Documented by...

Allison Jones

Allison is an Optimization Associate at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.