How including a preview of a survey question in a Facebook ad impacted clicks
Canadian Taxpayers Federation
A not-for-profit citizen's advocacy group dedicated to lower taxes, less waste and accountable government.
Experiment Summary
Ended On: 08/18/2022
We were running a survey offer on Facebook and wanted to develop a treatment to increase clickthrough rate. We wondered if replacing the stock imagery with a text-based ad that gave a preview of the first question on the survey would offer more clarity to ad viewers about what types of questions they would be asked to complete. We also changed the generic headline “Tell us what you think” to a more direct and specific CTA of “Take the survey” to offer additional clarity about the desired action we wanted them to take from the ad.
Research Question
We believe that adding clarity in a Facebook ad graphic for potential survey takers will achieve an increase in clickthrough rate.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.7% | ||
| T1: | Treatment #1 | 9.3% | 452.8% | 100.0% |
This experiment has a required sample size of 67 in order to be valid. Since the experiment had a total sample size of 35,810, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
452.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
What we found after running both versions of the ad was that the treatment design led to a 452% increase in clickthrough rate compared to the control with 100% level of confidence. Not only did we see a lift in the first step of the process, but we also saw a 307% increase in email acquisition. We weren’t able to just send more people to visit the landing page with the treatment, we were also able to generate more survey responses. In addition to the lifts in both traffic and conversion, we also saw a 99% decrease in our cost per acquisition. We believe the additional clarity in the text-based ad reduced friction and confusion about the desired next step which made it easier for ad viewers to begin taking the survey and created cognitive momentum to complete the full process.