How adding one required field affects donor conversion Experiment ID: #2112

Texas State Historical Association

Experiment Summary

Timeframe: 9/9/2015 - 9/18/2015

As part of their latest name acquisition campaign, the Texas State Historical Association had an instant donor conversion page once someone submitted their email address. Historically, we had trouble matching up donations to existing donors which had created troubles in the organization’s CRM system. We wanted to see if it was possible to acquire more information about their donors to make this process easier.

The ultimate goal was to simplify the gift entry process by acquiring the donor’s cell phone number. Having this piece of information would allow for duplicate donor records to be more easily identified.  Our concern was on whether this one field would help or hinder the ultimate conversion into a donor so we created two donation forms that would allow us to test the concept.

Research Question

What impact will requiring a cell phone have on the acquisition of donors?

Design

C: No Phone Number Field
T1: Required Cell Phone

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: No Phone Number Field 2.1%
T1: Required Cell Phone 1.2% -42.6% 95.5%

This experiment has a required sample size of 1,589 in order to be valid. Since the experiment had a total sample size of 3,349, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 42.6% decrease in conversion rate
× 0% increase in average gift

Key Learnings

The donation form that required a cell phone decreased the number of members acquired by 42.6%. Any form field added to a page will create a certain amount of friction in the visitor’s mind, even if it was option.  In this case, the visitors’ sensitivity to giving out their cell phone number is high. By requiring that piece of information, we added a certain amount of “cost” to the offer (in this case, membership) that was higher than the perceived value in many visitors’ minds.  This led to a reduction in the conversion on the page.


Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.


Pick your format of choice:

Related Experiments

This experiment is part of a series of experiments aimed at improving overall results. Take a look at some of the other iterations:

Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.