How a personalized slide out impacted email conversion rates
WUNC
Experiment Summary
Ended On: 08/02/2023
WUNC is North Carolina’s largest NPR station. In an effort to convert new web traffic into to subscribers we decided to test a more human approach slide out.
Research Question
We believe that making the slide out from an individual for new web visitors will achieve a higher email conversion because this is a personal invitation and expectations of the offer they will receive .
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.12% | ||
| T1: | Treatment #1 | 0.26% | 123.2% | 99.4% |
This experiment has a required sample size of 6,954 in order to be valid. Since the experiment had a total sample size of 27,076, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 123.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that using a personalized slide out from an individual for new web visitors resulted in a significant increase in email conversions. The treatment group, which received the personalized invitation, had a 123.2% increase in acquired emails compared to the control group.
This test only targeting new visitors to WUNC’s website so that we are reaching new names and not existing contacts.
This suggests that personalizing the communication and creating expectations of the offer can be highly effective in converting new web visitors into subscribers. This human approach resonated with the visitors and made them more likely to provide their email address.
In future experimentation, it is recommended to continue using this personalized slide out approach for new web visitors. This can be further refined by testing different variations of the personal invitation message, such as the tone and wording, to find the most effective combination. Additionally, it would be valuable to expand this experimentation to other conversion goals, such as donations or membership sign-ups, to see if the same human approach can be successful in those areas as well.