How a progressive form field impacts webinar conversion rate Experiment ID: #21203

NextAfter

Experiment Summary

Ended On: 5/29/2020

Internally, we’re wanting to find ways to get more data on the size of the organizations that subscribe to our emails. But we didn’t want to sacrifice conversion rate from our webinars for the sake of getting a little more data.

 

We wondered if we could add a field asking for their organization’s approximate annual revenue. We had previously tested into asking if they work for a non-profit vs for-profit. So we figured the least invasive way to ask for revenue was as a “progressive form field.” This means that the field only showed up if they checked that they worked for a nonprofit.

Research Question

Will adding a progressive form field to ask for approximate annual revenue impact conversions?

Design

C: Normal Form Fields
T1: Progressive Form Field Asking for Organization Size

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Normal Form Fields 65.4%
T1: Progressive Form Field Asking for Organization Size 63.5% -3.0% 42.6%

This experiment has a required sample size of 4,681 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

We saw a marginal shift in conversion rate – a drop of 3%. But this drop didn’t come anywhere close to reaching a significant level of confidence. It’s possible that running this for longer and getting more traffic to the page could prove that this decrease is in fact real – but the odds are low that this makes a significant impact over time.

 

We will continue to test the placement of this field, but knowing that the likelihood of a negative impact is very low. And the upside is that we get better segmentation and targeting for future outreach efforts.


Experiment Documented by...

Nathan Hill

Nathan is an Optimization Evangelist at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.