How changing Facebook ad copy for a resource offer impacted email sign ups for Pathway to Victory
First Baptist Dallas
Experiment Summary
Ended On: 02/09/2022
Pathway to Victory is a ministry who has built a fundraising program around the offer of premiums—books, pamphlets, brochures, etc. One of their most popular offers is a brochure focusing on the most popular Parables of Jesus.
For this experiment, we tested both a different copy treatment with a different approach to what the brochure offers, as well as images that showed the inside of the brochure.
Research Question
We believe that adding new copy and images for potential email sign-ups will achieve a change in clicks and sign-ups..
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.05% | ||
| T1: | New image and copy | 0.15% | 199.5% | 100.0% |
This experiment has a required sample size of 5,741 in order to be valid. Since the experiment had a total sample size of 1,438,372, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 199.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
By changing the copy and images so they gave a Facebook audience more insight into what they were going to be receiving, we saw a 199% increase in email sign up rates with 100% level of confidence.
Our hypothesis is that the control copy was too generic to interest people and get them to fill out a form on the landing page with all their information (the resource is mailed); once they got an ‘inside’ view of the content, they were much more likely to sign up to receive it.
This is an experiment we ran across multiple offers like this for Pathway to Victory, with similar results. The next iteration of this test would be to focus on isolating which component of the test (copy, image, etc) really led to the experiment results.