The Heritage Foundation

How a phone number field on a donation form affects donor conversion

Experiment ID: #2780

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 09/26/2017 - 10/31/2017

The Heritage Foundation had started using phone numbers to reach out and thank their donors for their gifts. Historically, the phone number was something that only came from direct mail donors. In order to increase the number of donors that had an available phone number on file, we wanted to see if adding a phone number field to the donation form would have an impact on donations.

Research Question

How will adding a phone number field impact donor conversion?

Design

C: No Phone Field
T1: Phone (optional)

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: No Phone Field 4.8%
T1: Phone (optional) 4.7% -1.7% 17.1%

This experiment has a required sample size of 542,448 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

In this situation, the insignificant result was exactly what we were hoping for. This proves that there is no significant impact to conversion by adding a phone number field and making it optional. Since it doesn’t impact conversion and provides us with additional data that will be helpful downstream, this has since been rolled out to all donation forms.


Experiment Documented by NextAfter

Question about experiment #2780

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.