How adding an in-line dear reader on blog and article pages impacted email acquisition
The Gospel Coalition
Experiment Summary
Timeframe: 10/27/2022 - 11/06/2022
Content offers for TGC were being run as display ads on the site. There were also email capture forms in the footer of each page offering people a relevant newsletter subscription based on the content they were consuming. But with high volume traffic to blog and article pages, we wondered if including an in-line free content offer that included the eBook cover image, some light value proposition, and a CTA would increase the number of emails acquired from website visitors. We split traffic for this experiment 25/75 with 75% of visitors receiving the treatment which included a dear reader at the end of the article or blog.
Research Question
We believe that an in-line content offer for blog and article visitors will achieve an increase in email acquisition.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.00% | $0.00 | ||
| T1: | Secular Creed Dear Reader | 1.1% | 27,919.7% | 100.0% | $0.00 |
This experiment has a required sample size of 540 in order to be valid. Since the experiment had a total sample size of 210,103, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 27,919.7% increase in conversion rate
× 0% increase in average gift
Key Learnings
What we found after running this experiment for 1 week was that there was a 27,919% increase in email conversion rate with 100% level of confidence. We saw valid increases across every channel, every device type, and every visitor segment. We quickly learned that web visitors responded well to this free content offer when it was featured in a highly visible placement on the web pages they were visiting. We have rolled the dear reader out to 100% of traffic and have several additional test opportunities to move forward on now that we have installed this feature. In the future, we want to test the impact of embedding the form itself so people can enter their name and email address directly on the site without clicking over to the landing page. We also want to test this against different content offers available so we can have something to offer for returning visitors who have already received this specific ebook.
Our most notable increases were for desktop visitors and returning visitors:
- Desktop visitors had a 12,780.6% increase in email acquisitions with a 100% level of confidence.
- Returning visitors had a 16,432.8% increase in email acquisitions with a 100% level of confidence.