Care Net

Impact of a time-sensitive Facebook Ad headline on conversion.

Experiment ID: #8279

Care Net

Experiment Summary

Ended On: 11/20/2018

Last year, one of our most successful ads was for our free Advent Devotional. This year, when we launched the ads we decided to test a new headline. In experiment #6574, we tested a clear CTA headline vs a question based headline and the CTA decreased conversion 47%. However, our control headline for Advent was a time-sensitive question, “Are you ready for Advent.” We wanted to know if the clear CTA would perform better than a season, time-sensitive headline like we used last year.

Research Question

Does a clear CTA headline perform better than a time-sensitive question in a Facebook ad?

Design

C: Seasonal Question
T1: Clear CTA

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Seasonal Question 1.0%
T1: Clear CTA 1.5%48.6% 99.1%

This experiment has a required sample size of 4,319 in order to be valid. Since the experiment had a total sample size of 16,093, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 48.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

Once again, a clear CTA performed better than a question. Even when the question was time-sensitive in nature (Advent was only a few weeks away). Remarkably, the increased conversion was almost the same as in our test in August of 47% increased conversion.


Experiment Documented by NextAfter

Question about experiment #8279

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.