The Heritage Foundation

How changing the landing page design affects revenue

Experiment ID: #1469

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 06/09/2015 - 06/23/2015

The Heritage Foundation had been using the same landing page design for their search engine marketing (SEM) donor campaigns for the past three years. During those three years, we had run multiple experiments on the main organizational donation page which had gradually evolved into something that looked completely different.

We were in the process of moving the SEM page to the same domain as the main donation form and realized that it would be a good time to validate that the old design was still effective for the SEM audience segment.  We created an experiment by duplicating the page content but displaying it on the two different page designs.

Research Question

Which landing page design produces the most donor conversions and revenue?


C: Design
T1: Secured.Heritage.Org Design


 Treatment NameRevenue per VisitorRelative DifferenceConfidenceAverage Gift
C: Design $2.65$88.19
T1: Secured.Heritage.Org Design $0.39-85.1% 99.9%$40.45

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 67.6% decrease in conversion rate
× 54.1% decrease in average gift
85.1% decrease in revenue

Key Learnings

The new design treatment caused an 85.1% drop in revenue by decreasing conversion rate by 67.6% and decreasing average gift by 54.1%. The simplified design used in the treatment was developed for a page with minimal copy. When we added the long value proposition text that had been developed for this specific landing page, it became difficult to read.

Landing pages must be designed so that they are easily scannable by the visitor. In this situation, the sub-headlines didn’t stand out enough and the lack of page border made the page seem longer than it was.  All of these elements caused enough friction in the mind of the donor to have a negative impact on both conversion rate and average gift.

Experiment Documented by NextAfter

Question about experiment #1469

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.