How form field friction affects email acquisition
Food for the Poor
Timeframe: 07/14/2016 - 08/10/2016
Food for the Poor was using Google ads to promote their “Verse of the Day” email. When the visitor signed up for the email, they asked for four pieces of information: First Name, Last Name, Email, and ZIP code. Though this was typical of email acquisition pages, they wanted to see if only asking for email and ZIP code would increase email acquisition rate. They developed a treatment that reduced the form fields to two and launched an A/B test to determine a winner.
Will reducing the number of form fields increase email acquisition rate?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|C:||Four form fields||0.26%|
|T1:||Two form fields||0.39%||48.3%||100.0%|
This experiment has a required sample size of 15,341 in order to be valid. Since the experiment had a total sample size of 122,534, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 48.3% increase in conversion rate
× 0% increase in average gift
The treatment produced a 48.3% increase in email conversion rate. This shows us that asking for less information can result in more signups. However, there is a trade-off to consider in that Food for the Poor will not be able to personalize their emails unless first and last name are eventually collected. For a simple email like “Verse of the Day”, such personalization may not be necessary.
Question about experiment #4699
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.