Impact of a time-sensitive Facebook Ad headline on conversion. Experiment ID: #9657

Care Net

Experiment Summary

Ended On: 11/20/2018

Last year, one of our most successful ads was for our free Advent Devotional. This year, when we launched the ads we decided to test a new headline. In experiment #6574, we tested a clear CTA headline vs a question based headline and the CTA decreased conversion 47%. However, our control headline for Advent was a time-sensitive question, “Are you ready for Advent.” We wanted to know if the clear CTA would perform better than a season, time-sensitive headline like we used last year.

Research Question

Does a clear CTA headline perform better than a time-sensitive question in a Facebook ad?

Design

C: Seasonal Question
T1: Clear CTA

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Seasonal Question 1.0%
T1: Clear CTA 1.5% 48.6% 99.1%

This experiment has a required sample size of 4,319 in order to be valid. Since the experiment had a total sample size of 16,093, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 48.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

Once again, a clear CTA performed better than a question. Even when the question was time-sensitive in nature (Advent was only a few weeks away). Remarkably, the increased conversion was almost the same as in our test in August of 47% increased conversion.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.