Reasons to Believe

How adding an ebook incentive in a survey impacts email acquisition

Experiment ID: #95279

Reasons to Believe

Experiment Summary

Timeframe: 05/17/2022 - 06/10/2022

RTB launched a new offer: Science and Faith Survey. Surveys have historically produced a lower sign up rate than other offers like ebooks. In order to provide additional incentive to provide contact information as well as provide additional value for the site visitor, we included a checkbox at the end of the survey that, if checked, will send them a free ebook. Our hypothesis is that this extra added incentive and value, will increase email acquisition.

Traffic to this offer was primarily driven by paid Facebook ads.

Research Question

We believe that adding an ebook incentive to a survey for survey takers will achieve a higher email acquisition rate .


C: Control
T1: Treatment


  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.17% $0.00
T1: Treatment 0.85% 383.9% 89.4% $0.00

This experiment has a required sample size of 875 in order to be valid. Since the experiment had a total sample size of 1,163, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

While this experiment did not validate, we did see a directional decrease

Notable changes to other metrics due to the experiment:

  • Donations experienced a 383.9% increase with a 89% level of confidence.
  • Revenue experienced a 709.5% increase with a 97% level of confidence.


Experiment Documented by NextAfter

Question about experiment #95279

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.