The Heritage Foundation

How visually highlighting security impacted donor conversion

Experiment ID: #6091

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 01/11/2017 - 02/08/2017

Based upon experiment results from similar organizations, we decided to launch an experiment across the primary donation forms for Heritage to see if visually reinforcing the security of the credit card form was able to lift donor conversion. The hypothesis was that by visually separating the form from the rest of the page, we could reduce the anxiety associate with entering a credit card online.

Research Question

Will visually highlighting the security of the credit card section reduce anxiety enough to increase donor conversion?

Design

C: Bare Fields
T1: Security Enhanced Fields

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Bare Fields 6.5%$91.70
T1: Security Enhanced Fields 6.8%4.4% 57.3%$90.59

This experiment has a required sample size of 59,340 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

This change did not seem to have any significant impact on the donor’s likelihood to make a gift. It may be possible to increase the visibility and the language around the security but, given this experiment result, it does not seem like it would be significant lift even with those changes. Given the current schedule of experiments, we feel that we would be better spent testing new technologies and radical changes than iterating on this concept.


Experiment Documented by NextAfter

Question about experiment #6091

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.