How adding an ebook incentive in a survey impacts email acquisition
Reasons to Believe
Experiment Summary
Timeframe: 05/17/2022 - 06/10/2022
RTB launched a new offer: Science and Faith Survey. Surveys have historically produced a lower sign up rate than other offers like ebooks. In order to provide additional incentive to provide contact information as well as provide additional value for the site visitor, we included a checkbox at the end of the survey that, if checked, will send them a free ebook. Our hypothesis is that this extra added incentive and value, will increase email acquisition.
Traffic to this offer was primarily driven by paid Facebook ads.
Research Question
We believe that adding an ebook incentive to a survey for survey takers will achieve a higher email acquisition rate .
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.17% | $0.00 | ||
| T1: | Treatment | 0.85% | 383.9% | 89.4% | $0.00 |
This experiment has a required sample size of 875 in order to be valid. Since the experiment had a total sample size of 1,163, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
While this experiment did not validate, we did see a directional decrease
Notable changes to other metrics due to the experiment:
- Donations experienced a 383.9% increase with a 89% level of confidence.
- Revenue experienced a 709.5% increase with a 97% level of confidence.