How paginating a survey affects completion rate

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 01/27/2020 - 02/22/2020

Hillsdale College was running a survey about the state of higher education. They had moved from SurveyMonkey to Unbounce to streamline their survey creation process, and were curious if they could get more people to complete the survey by breaking it into smaller sections that were paginated. Their hypothesis was that paginating the survey would result in more “momentum” as people completed multiple smaller surveys, rather than one long survey on the page. They created both versions in the new platform and launched an A/B test to see how it affected completion rate. 

Research Question

How will paginating a survey affect completion rate?

Design

C: Survey Page 1

T1: Survey (Paginated Version)

Results

This experiment has a required sample size of 103 in order to be valid. Since the experiment had a total sample size of 18,385, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 24.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

The paginated survey generated 24.8% more completions than the single page survey, which indicated that the “multiple short surveys” approach helped decrease the perceived friction that a long survey created. This resulted in more people seeing a donation page, which resulted in more donations.