How giving a user more choice through carousel sponsorship ads affects clickthrough rate
Prison Fellowship International
Experiment Summary
Ended On: 08/24/2021
Prison Fellowship International was continuing to promote child sponsorship through Facebook ads that pulled children dynamically from their website into the ad. They wondered if their single-child approach (one child per ad) put too much pressure on the visitor to sponsor that specific child, rather than giving them a choice. Since they had the choice to use Facebook’s carousel ad feature, they decided to test to see if showing multiple children would give the user more choice and increase the potential that they would sponsor a child.
They launched two campaigns in an A/B test to determine a winner.
Research Question
We believe that giving more choice through carousel ads for potential sponsors will achieve an increase in traffic.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 2.7% | ||
| T1: | Treatment #1 | 3.4% | 27.0% | 100.0% |
This experiment has a required sample size of 4,332 in order to be valid. Since the experiment had a total sample size of 140,395, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
27.0% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment carousel ads increased link clicks by 27%, which drove much more traffic to the site. However, conversion rate on the site was too low to determine if it drove a more motivated visitor, because the results are statistically significant, we can know that it does increase user engagement with the ad and does drive more clicks per dollar, resulting in a 36% decrease in cost per click.