How a reintroducing clickable looking text in an ad impacted donor conversion rate
Americans for Prosperity
Experiment Summary
Ended On: 04/01/2021
In a recent study, Americans for Prosperity tested their way into a simplified ad experience with an ad image promoting a petition funnel that used what appeared like buttons (saying “Yes” and “No” in response to a question).
Unfortunately, this ad image was eventually rejected by Facebook due to it not adhering to ad policy guidelines that specifically prohibit images that have “fake buttons” on them to generate high clickthrough rates.
Therefore, it forced us to try another approach — one that looked like a banner, and another that looked like “clickable text” in response to the question the ad posed.
Which approach would produce the best donor conversion rate?
Research Question
We believe that clickable text in an ad image for prospective donors will achieve an increase in donor conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Banner Approach | 0.02% | ||
| T1: | Clickable Text Approach | 0.05% | 238.0% | 99.5% |
This experiment has a required sample size of 17,916 in order to be valid. Since the experiment had a total sample size of 81,643, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 238.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
With a 99.5% level of confidence, we observed a 238% increase in donor conversion rate when using the “clickable text” approach to the question-based ad promotion.
We also observed the following performance against the banner (control) approach:
- +345% in ad clickthrough rate
- +261% in email signup rates
This ad image approach captured most of the essence of the original “fake button” approach we introduced and tested our way into. So long as your ad image is using what doesn’t appear to be a button, it appears that you can meet the standards of the Facebook ad policy, while also maintaining the results that “fake button” approach would produce.