How a GDPR compliant form impacted email acquisition Experiment ID: #9000
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Timeframe: 5/10/2018 - 5/18/2018
With the upcoming roll out of the General Data Protection Regulation (GDPR), we wanted to get an idea of the kind of impact that the explicit opt-in clause would have on email acquisition. To do this, we ran an A/B test where we added a required checkbox that informed visitors they would get both the offer and email updates in the future.
What impact will a GDRP-compliant opt-in have on email acquisition?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 357 in order to be valid. Since the experiment had a total sample size of 3,040, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 15.4% decrease in conversion rate
× 0% increase in average gift
After running for a full week, we saw that the explicit opt-in checkbox resulted in a statistically significant 15% drop in email acquisition. Our hypothesis is that the additional checkbox resulted in increased friction in the process and the vague language around “occasional updates” added anxiety to the decision which both had a negative impact on conversion.
Given the importance of this regulation, this is an area that we will need to continue to test and optimize to find the ideal tactic for the explicit opt-in requirement.