How adding an additional cultivation opportunity affects email acquisition
Hoover Institution
Experiment Summary
Timeframe: 05/07/2018 - 05/11/2018
Hoover Institution was running their Make America Exceptional Again offer, and wanted to know the effects of offering a checkbox to sign up for the HDR newsletter by the bottom of the form. All copy remained the same.
Research Question
Does adding a checkbox to receive additional content at the end of a form affect conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Newsletter Opt-In | 24.5% | ||
| T1: | Newsletter Opt-In | 25.0% | 2.0% | 13.2% |
This experiment has a required sample size of 57,976 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
This experiment did not experience a statistically valid lift. However, this experiment not statistically validating is just as important as if we had seen a lift. This means we can increase the subscribers to the newsletter without adversely impacting acquisition.
The goal of these email acquisition campaigns is to create a file of prospective donors that can also be converted downstream. The best way to do that is through ongoing cultivation.
By getting a new subscriber to opt-in to the ongoing newsletter cultivations, we are putting them into a communication flow that will increase the likelihood of downstream conversion. Since they are having to physically opt-in to the series, it will also increase their perceived value of the communications they will receive.