NextAfter

How a different value proposition affects click rate in a Facebook Ad

Experiment ID: #11738

NextAfter

Experiment Summary

Timeframe: 08/08/2016 - 08/12/2016

We released a research study on nonprofit value proposition and advertised it through Facebook Ads to people affiliated with NextAfter including emails on file and website visitors.

There are many different aspects of the value proposition that we could focus on for the promotion of the study, so we wanted to test 3 primary value propositions to see what resonated the most with our audience.

The control focused on the depth of the study. Treatment 1 focused on the 4 key components of a value proposition. And treatment 2 focused on the end result of an effective value proposition.

Research Question

Which value proposition will drive the most clicks from a Facebook Ad?

Design

C: 127 Nonprofits
T1: 4 Key Components
T2: Names, Donors, and Dollars

Results

  Treatment Name Click Rate Relative Difference Confidence
C: 127 Nonprofits 0.53%
T1: 4 Key Components 0.30% -43.4% 36.9%
T2: Names, Donors, and Dollars 1.9% 254.8% 98.8%

This experiment has a required sample size of 611 in order to be valid. Since the experiment had a total sample size of 1,932, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    254.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

For the audience that has already engaged with us in some way, it was 254% more effective to focus on the results of more names, donors, and dollars. This will inform the way the we promote and communicate this study to our website traffic and engaged contacts.

This also points to an underlying motivation of our engaged contacts: these people are looking for results in their fundraising.


Experiment Documented by Dan VanMilligan
Dan VanMilligan is Implementation and Optimization Specialist at NextAfter.

Question about experiment #11738

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.