How a different value proposition affects conversion in a Facebook Ad Experiment ID: #9237


Experiment Summary

Ended On: 7/23/2018

In this experiment, we started advertising an upcoming webinar wondering which value proposition focus would be most effective: focusing on the comprehensiveness of the research study, or focusing on the key insights and questions that the study will answer.

Audience: Nonprofit News Source (NP Times, Chronicle)

Research Question

Will a value proposition focused on the key insights from the study increase registrations?


C: Comprehensive Benchmark
T1: Key Insights


Treatment Name Conv. Rate Relative Difference Confidence
C: Comprehensive Benchmark 0.18%
T1: Key Insights 0.81% 351.3% 96.3%

This experiment has a required sample size of 1,210 in order to be valid. Since the experiment had a total sample size of 1,925, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 351.3% increase in conversion rate
× 0% increase in average gift

Key Learnings

Key insights increased conversion by 351.3% for the “Nonprofit News Sources” audience. This is the opposite result of what we saw when targeting traffic. This could imply that the nonprofit news source audience is more of an executive or leadership level employee that is focused on higher level results. More testing will be needed to determine what the key motivators are for this audience.

Experiment Documented by...

Nathan Hill

Nathan is an Optimization Evangelist at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.