Canadian Taxpayers Federation

How including a preview of a survey question in a Facebook ad impacted clicks

Experiment ID: #104471

Canadian Taxpayers Federation

A not-for-profit citizen's advocacy group dedicated to lower taxes, less waste and accountable government.

Experiment Summary

Ended On: 08/18/2022

We were running a survey offer on Facebook and wanted to develop a treatment to increase clickthrough rate. We wondered if replacing the stock imagery with a text-based ad that gave a preview of the first question on the survey would offer more clarity to ad viewers about what types of questions they would be asked to complete. We also changed the generic headline “Tell us what you think” to a more direct and specific CTA of “Take the survey” to offer additional clarity about the desired action we wanted them to take from the ad.

Research Question

We believe that adding clarity in a Facebook ad graphic for potential survey takers will achieve an increase in clickthrough rate.


C: Control
T1: Treatment #1


  Treatment Name Click Rate Relative Difference Confidence
C: Control 1.7%
T1: Treatment #1 9.3% 452.8% 100.0%

This experiment has a required sample size of 67 in order to be valid. Since the experiment had a total sample size of 35,810, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    452.8% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

What we found after running both versions of the ad was that the treatment design led to a 452% increase in clickthrough rate compared to the control with 100% level of confidence. Not only did we see a lift in the first step of the process, but we also saw a 307% increase in email acquisition. We weren’t able to just send more people to visit the landing page with the treatment, we were also able to generate more survey responses. In addition to the lifts in both traffic and conversion, we also saw a 99% decrease in our cost per acquisition. We believe the additional clarity in the text-based ad reduced friction and confusion about the desired next step which made it easier for ad viewers to begin taking the survey and created cognitive momentum to complete the full process.

Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #104471

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.