How the order of application questions affects name acquisition
Global Volunteer Network
Experiment Summary
Timeframe: 09/26/2012 - 10/03/2012
Selecting which project to volunteer for is one of the most thought provoking decisions a volunteer makes when signing up the Global Volunteer Network. The original sign up process had volunteers selecting this information at the beginning of their application. It was out hypothesis that putting such an important question at the start caused a significant drop off in the sign up process. We wanted to run an experiment to determine what moving the question further down the process would do for the number of volunteers acquired.
Research Question
By moving the project selection later in the process, we are allowing volunteers to become mentally invested in the application so that the selection process does not cause as many drop offs.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Select Project in Step 1 | 3.5% | ||
| T1: | Select Project in Step 2 | 5.3% | 51.5% | 99.9% |
This experiment has a required sample size of 997 in order to be valid. Since the experiment had a total sample size of 5,661, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 51.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
Moving the project selection later in the application process resulted in an increase of 51.5% in volunteers acquired. The volunteers had already filled out their basic biographical information so were already invested in their application. When the project selection question did appear, they were already committed to the process so the challenging question did not result in as many dropoffs.