How a popup impacted email newsletter sign ups
WUNC
Experiment Summary
Ended On: 06/05/2023
WUNC is a North Carolina Public Radio Station. To convert web traffic to email subscribers, we tested a slide out with fields to sign up for the weekly What’s Trending Newsletter.
The existing, only email newsletter sign-up location is on the homepage after scrolling over half way down the page. No offers are available on other landing pages.
Research Question
We believe that by adding a email sign up slide out for website visitors will achieve a higher conversion of email sign ups because the opportunity is presented in a clearer fashion.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.05% | ||
| T1: | Treatment #1 | 0.34% | 592.0% | 100.0% |
This experiment has a required sample size of 1,748 in order to be valid. Since the experiment had a total sample size of 16,092, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 592.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that the addition of an email sign up slide out can significantly increase the conversion of email sign ups. The treatment group had a 395.5% increase in emails acquired for all traffic compared to the control group. This result was statistically significant with 100.0% confidence.
For future experiments we will test into further clarity on the slide out, explaining what the subscriber is signing up both driving clarity and incentive.