How a GDPR compliant form impacted email acquisition Experiment ID: #9000

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 5/10/2018 - 5/18/2018

With the upcoming roll out of the General Data Protection Regulation (GDPR), we wanted to get an idea of the kind of impact that the explicit opt-in clause would have on email acquisition. To do this, we ran an A/B test where we added a required checkbox that informed visitors they would get both the offer and email updates in the future.

Research Question

What impact will a GDRP-compliant opt-in have on email acquisition?

Design

C: Control
T1: GDPR Version

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 47.1%
T1: GDPR Version 39.8% -15.4% 100.0%

This experiment has a required sample size of 357 in order to be valid. Since the experiment had a total sample size of 3,040, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 15.4% decrease in conversion rate
× 0% increase in average gift

Key Learnings

After running for a full week, we saw that the explicit opt-in checkbox resulted in a statistically significant 15% drop in email acquisition. Our hypothesis is that the additional checkbox resulted in increased friction in the process and the vague language around “occasional updates” added anxiety to the decision which both had a negative impact on conversion.

Given the importance of this regulation, this is an area that we will need to continue to test and optimize to find the ideal tactic for the explicit opt-in requirement.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.