How the kind of content offer delivery mechanism impacts conversion rate
Care Net
Experiment Summary
Timeframe: 01/24/2018 - 02/01/2018
Care Net has a variety of content offers that they offer to people who come to their website. The offers were being delivered to people via an exit-intent pop-up on their site. This means that as a person would start to leave the page, a pop-up would appear on their screen with one of thecontent offers. We hypothesized that with the high volume of mobile traffic on their website, a different delivery mechanism might help increase name acquisition from their site since exit-intent pop-ups don’t work or show up on mobile.
Research Question
Which delivery mechanism would increase name acquisition?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Exit-intent | 1.1% | ||
| T1: | Slideout | 3.2% | 194.8% | 100.0% |
This experiment has a required sample size of 371 in order to be valid. Since the experiment had a total sample size of 2,338, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 194.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
After a little over a week, we saw a significant difference in conversion between the exit-intent pop-up and slide out. The slide-out increased conversion by 194.8%! After analyzing the data, we want to see the kind of impact this kind of experiment would have on desktop vs mobile. This would require us to run the experiment longer to get a larger sample size. The larger sample size would also help reinforce the significant impact and lift that this kind of delivery mechanism can have on acquisition as a whole.