How a progressive form field impacts webinar conversion rate

NextAfter

Experiment Summary

Ended On: 05/29/2020

Internally, we’re wanting to find ways to get more data on the size of the organizations that subscribe to our emails. But we didn’t want to sacrifice conversion rate from our webinars for the sake of getting a little more data.

We wondered if we could add a field asking for their organization’s approximate annual revenue. We had previously tested into asking if they work for a non-profit vs for-profit. So we figured the least invasive way to ask for revenue was as a “progressive form field.” This means that the field only showed up if they checked that they worked for a nonprofit.

*This was the 2nd of an identical test to see if greater volume would impact significance.

Research Question

Will adding a progressive form field to ask for approximate annual revenue impact conversions?

Results

This experiment has a required sample size of 3,447 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

We saw a marginal shift in conversion rate – a drop of 3.9%. But this drop didn’t come anywhere close to reaching statistical significance. This was the 2nd of an identical test after seeing an “invalid” result the first time. We thought that more volume could give a stronger indication of impact. The combined results of this experiment yield the same result…a 3.6% decrease, but with an invalid result. If we determine that the benefit of getting this user-reported revenue data more valuable than the slight risk of small decrease in conversion, we’ll roll this treatment out.  

---