How a reintroducing clickable looking text in an ad impacted donor conversion rate | NextAfter
Americans for Prosperity

How a reintroducing clickable looking text in an ad impacted donor conversion rate

Experiment ID: #56297

Americans for Prosperity

Experiment Summary

Ended On: 04/01/2021

In a recent study, Americans for Prosperity tested their way into a simplified ad experience with an ad image promoting a petition funnel that used what appeared like buttons (saying “Yes” and “No” in response to a question).

Unfortunately, this ad image was eventually rejected by Facebook due to it not adhering to ad policy guidelines that specifically prohibit images that have “fake buttons” on them to generate high clickthrough rates.

Therefore, it forced us to try another approach — one that looked like a banner, and another that looked like “clickable text” in response to the question the ad posed.

Which approach would produce the best donor conversion rate?

Research Question

We believe that clickable text in an ad image for prospective donors will achieve an increase in donor conversion rate.

Design

C: Banner Approach
T1: Clickable Text Approach

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Banner Approach 0.02%
T1: Clickable Text Approach 0.05% 238.0% 99.5%

This experiment has a required sample size of 17,916 in order to be valid. Since the experiment had a total sample size of 81,643, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 238.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 99.5% level of confidence, we observed a 238% increase in donor conversion rate when using the “clickable text” approach to the question-based ad promotion.

We also observed the following performance against the banner (control) approach:

  • +345% in ad clickthrough rate
  • +261% in email signup rates

This ad image approach captured most of the essence of the original “fake button” approach we introduced and tested our way into. So long as your ad image is using what doesn’t appear to be a button, it appears that you can meet the standards of the Facebook ad policy, while also maintaining the results that “fake button” approach would produce.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #56297

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.