How the right offer affects emails acquired
NextAfter
Experiment Summary
Timeframe: 10/28/2015 - 11/10/2015
When we first launched the new NextAfter.com website, we had some internal debate about the best offer to use in the footer of the site. Historically, we had offered a subscription to our weekly newsletter, This vs. That, which allowed subscribers to receive one experiment per week from the Research Library.
Since this was our main website, we hypothesized that maybe the more general offer of “Staying Informed” would work better for the visitors. We decided to set up a test that would allow us to evaluate these two different offers.
Research Question
Which offer in the footer of the NextAfter website will lead to the greatest number of emails acquired?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | This vs. That | 0.00% | ||
| T1: | Stay Informed | 4.1% | 100.0% | 98.8% |
This experiment has a required sample size of 85 in order to be valid. Since the experiment had a total sample size of 321, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 100.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
We discovered that the “Stay Informed” offer led to an infinitely better number of emails captured (none versus some is by definition infinitely better). While the This vs. That newsletter is still one of our more popular offers, a more general contact form was a better fit given the motivation of the people visiting the website.