How a simpler, more direct value proposition affects clickthrough rate on a Facebook ad Experiment ID: #6477

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 3/12/2017 - 4/10/2017

The Heritage Foundation was promoting a quiz around the “Gadsden Flag” that led to a membership offer with the flag as a premium. They had created two versions of a Facebook ad, as they had seen some research that longer, more engaging copy could increase clickthrough and conversion rate. The control, in this case, had much longer copy that talked about the history of the flag and made an offer to the reader to test their knowledge. The treatment copy was much shorter, and asked a direct question to the reader: “How much do you know about the Gadsden Flag?”

They split the traffic to determine which ad attracted more clicks.

Research Question

How does a shorter, more direct value proposition affect clickthrough rate on a Facebook ad?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: ad2 - long copy
T1: ad1

Results

Treatment Name Click Rate Relative Difference Confidence
C: ad2 - long copy 1.1%
T1: ad1 2.4% 122.1% 100.0%

This experiment has a required sample size of 891 in order to be valid. Since the experiment had a total sample size of 1,097,283, and the level of confidence is above 95%, the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    122.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The shorter ad with the direct question produced a 122.1% increase in clickthrough rate, which showed that the elements of this ad increased the clarity of the value proposition. Longer copy can be more effective, but only if if clarifies the value proposition. In addition, the direct question to provoke the reader to respond by taking the intended action.

This success prompted further testing to isolate the variables of length and the direct question to understand how each of these played into the success of the ad.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.