How language specific to existing contacts impacted Facebook clicks retest
First Baptist Dallas
Experiment Summary
Timeframe: 05/17/2023 - 05/20/2023
First Baptist Dallas is a church in the heart of downtown Dallas, Texas. Leading up to Sunday services, First Dallas utilizes Facebook to share what the sermon will be and invites people to join in person.
This campaign also targets existing First Dallas contacts through the HubSpot/Facebook integration. We decided to test and see if copy that is specific to this audience would increase clicks on the ads.
Research Question
We believe that by making ad copy specific to FBD contacts for existing contacts on Facebook will achieve a higher conversion in clicks because they are more personalized to their relationship with First Dallas. because they are more personalized to their relationship with First Dallas.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | FBD 350 - Join Us Reach - May 21 member control | 0.12% | ||
| T1: | FBD 350 - Join Us Reach - May 21 member treatment | 0.14% | 20.9% | 97.4% |
This experiment has a required sample size of 166,695 in order to be valid. Since the experiment had a total sample size of 434,052, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
20.9% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this A/B experiment for First Baptist Dallas is that creating ad copy specific to their existing contacts on Facebook can drive higher conversion in clicks. The treatment resulted in a 20.9% increase in Clicks for All Traffic with a high statistical level of confidence of 97.4%. Additionally, Cost per Click experienced a 13.5% decrease with a 100% level of confidence, and there was an increase in clicks from both mobile and tablet visitors.
Based on these findings, First Baptist Dallas should continue to personalize their ad copy for their existing contacts on Facebook and consider testing different variations of personalized copy in future experiments. They should also explore other ways to personalize their messaging to their audience, such as tailoring images or targeting specific demographics. By continuing to experiment and optimize their Facebook ad campaigns, First Baptist Dallas can maximize their conversion and engagement rates.