How a GDPR compliant form impacted email acquisition

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 05/10/2018 - 05/18/2018

With the upcoming roll out of the General Data Protection Regulation (GDPR), we wanted to get an idea of the kind of impact that the explicit opt-in clause would have on email acquisition. To do this, we ran an A/B test where we added a required checkbox that informed visitors they would get both the offer and email updates in the future.

Research Question

What impact will a GDRP-compliant opt-in have on email acquisition?

Design

C: Control

T1: GDPR Version

Results

This experiment has a required sample size of 357 in order to be valid. Since the experiment had a total sample size of 3,040, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 15.4% decrease in conversion rate
× 0% increase in average gift

Key Learnings

After running for a full week, we saw that the explicit opt-in checkbox resulted in a statistically significant 15% drop in email acquisition. Our hypothesis is that the additional checkbox resulted in increased friction in the process and the vague language around “occasional updates” added anxiety to the decision which both had a negative impact on conversion.

Given the importance of this regulation, this is an area that we will need to continue to test and optimize to find the ideal tactic for the explicit opt-in requirement.