How changing the landing page design affects revenue
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 06/09/2015 - 06/23/2015
The Heritage Foundation had been using the same landing page design for their search engine marketing (SEM) donor campaigns for the past three years. During those three years, we had run multiple experiments on the main organizational donation page which had gradually evolved into something that looked completely different.
We were in the process of moving the SEM page to the same domain as the main donation form and realized that it would be a good time to validate that the old design was still effective for the SEM audience segment. We created an experiment by duplicating the page content but displaying it on the two different page designs.
Research Question
Which landing page design produces the most donor conversions and revenue?
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | JoinHeritage.org Design | $2.65 | $88.19 | ||
| T1: | Secured.Heritage.Org Design | $0.39 | -85.1% | 99.9% | $40.45 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 67.6% decrease in conversion rate
× 54.1% decrease in average gift
85.1% decrease in revenue
Key Learnings
The new design treatment caused an 85.1% drop in revenue by decreasing conversion rate by 67.6% and decreasing average gift by 54.1%. The simplified design used in the treatment was developed for a page with minimal copy. When we added the long value proposition text that had been developed for this specific landing page, it became difficult to read.
Landing pages must be designed so that they are easily scannable by the visitor. In this situation, the sub-headlines didn’t stand out enough and the lack of page border made the page seem longer than it was. All of these elements caused enough friction in the mind of the donor to have a negative impact on both conversion rate and average gift.