How clarity affects clickthrough rate on a Facebook ad
American Cornerstone Institute
Experiment Summary
Ended On: 11/08/2023
Ben Carson’s American Cornerstone Institute was running ads promoting their Little Patriots Curriculum. On review, the ads that had performed best seemed to have a mismatch of creative and text. “Liberty the Eagle” was featured in the image, while the text asked the reader to pledge to teach kids founding principles.
Liberty the Eagle is a character in the Little Patriots curriculum, but that’s only encountered after they sign up — which means at the point of seeing the ad, the reader has no idea who Liberty is. They decided to replace this with a picture of Dr. Carson, a well-known figure. Once they did that, they realized they could change the “sign the pledge” ask to a “statement of belief” ask that is followed by the “sign the pledge” ask.
They launched two copy variants of the new ads to their audiences to see what worked best.
Research Question
We believe that increasing the clarity of the ask for Facebook visitors will achieve an increase in clickthrough rate because they understand what we are asking them to do..
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 4.8% | ||
| T1: | Treatment #1 | 11.8% | 147.2% | 100.0% |
| T2: | Treatment #2 | 15.2% | 218.6% | 100.0% |
This experiment has a required sample size of 105 in order to be valid. Since the experiment had a total sample size of 1,077,932, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 147.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
Key learnings from this experiment include:
1. The importance of aligning the creative and text in an ad: The mismatch between the image of “Liberty the Eagle” and the text asking for a pledge to teach kids founding principles resulted in lower clickthrough rates. This suggests that the audience may not have understood the ask or the connection between the image and text.
2. Clarity in the call-to-action: Increasing the clarity of the ask for Facebook visitors led to a significant increase in clicks. This emphasizes the importance of clearly communicating the desired action or response from the audience.
3. The significance of statistical confidence: The treatment group showed a 218.6% increase in donations with a 100.0% statistical level of confidence. This high level of statistical confidence suggests that the results are reliable and not due to random chance.
To utilize these learnings in future experimentation, it is crucial to ensure that the creative and text in ads are aligned and coherent. Additionally, clear and explicit calls-to-action should be employed to improve engagement and conversions. It is also important to continue using statistical analysis to determine the significance of the results and validate the effectiveness of the changes made.