Care Net

The impact of a Yes/No Image Ad on Facebook for volunteer recruitment

Experiment ID: #58790

Care Net

Experiment Summary

Ended On: 04/29/2021

We wanted to test whether using an image that enabled viewers on Facebook to feel like they were providing immediate feedback would increase conversion. So, we designed a static image with the question of our Facebook Survey on it and a big Yes and a big No option for them to “click” on. While not serving as working buttons, we hypothesized that merely simulating a button would be enough to increase conversion rate.

We tested this against a short video ad created using Facebook’s image/video tool.

Research Question

We believe that enabling immediate feedback via the image design for viewers on Facebook will achieve greater volunteer conversion rates .

Design

C: Control
T1: Treatment #1

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.06%
T1: Treatment #1 0.16%191.6% 100.0%

This experiment has a required sample size of 6,454 in order to be valid. Since the experiment had a total sample size of 89,400, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 191.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

The feedback ad improved conversion on every metric. Not only did we increase volunteer signups by 191%, we decreased the cost per signup 76% (from $37.95 to 8.93) and the cost per click 81% ($0.37 to $0.07). WE are testing this now on our monthly donor campaign to see if it has similar results.


Experiment Documented by NextAfter

Question about experiment #58790

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.