How adding more clarity and recency copy to a headline on a landing page affects email conversions
e3 Partners
Experiment Summary
Timeframe: 06/07/2022 - 08/27/2022
e3 Partners equips believers to take ownership of the Great Commission. They send believers into local and global communities to evangelize the lost, and they partner with believers to establish new churches across the globe. For this experiment, e3 wanted to test adding clarity and recency copy to their headline on an email acquisition landing page. The control headline had been used for a while and had good results, but after seeing other experiments that tested tried and true content offer copy, they decided to try it on their previously good performing content offer – prayers for the overwhelmed. Since this test ran during a period of societal uncertainty with recent tragedies in the news, inflation fears, and world unrest, we felt it was a prime opportunity to adjust the headline copy to reflect these worldwide events, while still remaining generic enough to stay evergreen.
Research Question
We believe that adding more clarity into a landing page headline, and aligning the message with recent events will result in higher email conversions.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.07% | $0.00 | ||
| T1: | Treatment | 0.21% | 205.8% | 69.0% | $0.00 |
This experiment has a required sample size of 5,235 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
While we did not have enough traffic to validate the experiment, we did see a 206% increase in conversions for the treatment with a 69% level of confidence. This was a very small sample size, but we did not see a negative effect on any of the channels. Due to this, we decided to role this out oas the new control. The next steps in the experiment might be to test some tactical elements, like testimonials or social proof that aligns with the recency copy.