How aligning the image and value proposition for a content offer affect email acquisition
The Navigators
Experiment Summary
Timeframe: 06/01/2021 - 06/12/2021
The Navigators were promoting their Digital Discipleship Journey resource through Facebook ads. Their cost-per-name was above what they projected, and they wanted to see how they could optimize their results. A previous test had shown that an image that was more specific to the offer greatly increased conversion rate and CPA, so they wanted to apply that to the landing page as well. Additionally, they wanted to test the content on the landing page to see if increasing clarity of the offer and removing vertical space could increase conversion.
They created two treatments and ran an A/B/C test to determine a winner.
Research Question
We believe that aligning the image with the offer and increasing the clarity of the value proposition for landing page visitors will achieve an increase in emails acquired.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 29.1% | $0.00 | ||
| T1: | Image change only (B) | 29.8% | 2.3% | 24.1% | $0.00 |
| T2: | Copy + image change (version C) | 29.0% | -0.4% | 4.0% | $0.00 |
This experiment has a required sample size of 40,855 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
Ultimately, neither landing page made a significant difference in conversion rate. There was directional evidence that the new image provided a slight bump to conversion rate, but it wasn’t near significance.
The 29% conversion rate on the landing page is still low compared to other Navigators offers, but this test suggests that the issue is upstream from the landing page—perhaps in the targeting or in the advertising.