The Heritage Foundation

How different button types in a giving array affect donation conversion

Experiment ID: #1812

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 10/31/2013 - 11/06/2013

The Heritage Foundation utilizes a giving system that allows for the creation of a prepopulated gift array. We can make suggested gift options to the donors as a way to increase the average gift. Their giving system’s default display for giving arrays utilize radio buttons.  This feature is helpful but the method of display can present a challenge for some visitors. It was our hypothesis that radio buttons might be difficult to click on a mobile phone and might not be as visible as other elements on the screen.

We wanted to find keep the impact that suggested gift arrays provide while creating a better user experience for the donor. As are result, we created a new treatment in one of their fundraising campaigns that utilized colorful buttons in place of the radio buttons.

Research Question

Will the colorful buttons for a giving array increase the conversion rate of the page over radio buttons?

Design

C: Radio Buttons
T1: Buttons

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Radio Buttons 13.6%$46.89
T1: Buttons 20.0%46.3% 99.9%$50.00

This experiment has a required sample size of 255 in order to be valid. Since the experiment had a total sample size of 1,746, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 46.3% increase in conversion rate
× 6.6% increase in average gift
56.0% increase in revenue

Key Learnings

The buttons were able to increase the conversion rate by 46.3%. While these buttons are a relatively small part of the donor experience, the selection of their gift is critical in the giving funnel. By making this easier to accomplish and more visually impactful, we were able to increase the number of donors that made a gift


Experiment Documented by Tim Kachuriak
Tim Kachuriak is Chief Innovation and Optimization Officer of NextAfter.

Question about experiment #1812

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.