How a phone number field on a donation form affects donor conversion
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 09/26/2017 - 10/31/2017
The Heritage Foundation had started using phone numbers to reach out and thank their donors for their gifts. Historically, the phone number was something that only came from direct mail donors. In order to increase the number of donors that had an available phone number on file, we wanted to see if adding a phone number field to the donation form would have an impact on donations.
Research Question
How will adding a phone number field impact donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | No Phone Field | 4.8% | ||
| T1: | Phone (optional) | 4.7% | -1.7% | 17.1% |
This experiment has a required sample size of 542,448 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
In this situation, the insignificant result was exactly what we were hoping for. This proves that there is no significant impact to conversion by adding a phone number field and making it optional. Since it doesn’t impact conversion and provides us with additional data that will be helpful downstream, this has since been rolled out to all donation forms.