How a different offer affects email acquisition Experiment ID: #30421


Experiment Summary

Timeframe: 8/20/2020 - 8/28/2020

In this experiment, we had a list of attendees from an event we sponsored. We want more people to sign up for our own event, but ultimately, we just want to get them to opt-in to communication so we can develop a relationship.

So we wondered…is it more effective to send this list an email about our conference and ask them to register for a free ticket? Or should offer them something of immediate value like a research study?

Research Question

Which offer will lead to more emails acquired?


C: NIO Summit Offer
T1: Donation Page Research Offer


Treatment Name Conv. Rate Relative Difference Confidence
C: NIO Summit Offer 0.33%
T1: Donation Page Research Offer 0.56% 72.5% 78.9%

This experiment has a required sample size of 6,095 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

The donation page offer increased conversion by 72%, although it only reached a 78% level of confidence. Although this experiment didn’t reach the level of confidence we would like to see, it’s an indicator that NIO Summit might not be the best initial email acquisition point for an email prospecting campaign.

The donation page offer did increase clicks by 58% and reached statistical validity.

Experiment Documented by...

Nathan Hill

Nathan is an Optimization Evangelist at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.