How the delivery of the offer can impact email acquisition
Illinois Policy Institute
Experiment Summary
Timeframe: 02/15/2017 - 03/08/2017
Illinois Policy Institute’s website sees significant volumes of daily traffic. They also have a weekly newsletter that contains investigative reporting and other top headlines from around the state of Illinois. Historically, this newsletter could be subscribed to by clicking on a link found in the footer of every page.
We had the hypothesis that we could increase email acquisition by making the offer easier to find by brining it up higher in the page. We decided to test two new methods to see which would have the greatest impact, an inline email acquisition offer or a slide out that appeared as a visitor scrolled down the page.
Research Question
Which email offer would produce the highest email acquisition rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Direct Offer | 0.14% | ||
| T1: | Inline Newsletter Offer | 0.61% | 350.1% | 100.0% |
| T2: | Slideout Newsletter Offer | 0.78% | 470.2% | 100.0% |
This experiment has a required sample size of 1,422 in order to be valid. Since the experiment had a total sample size of 68,404, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 350.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
Both the inline email acquisition and slide out were able to lift the email acquisition of the site. The slide out offer proved to be the most effective with a 470% lift. Our hypothesis is that this was due to the offer being more disruptive to the reading process which arrested the visitors’ attention.