Texas State Historical Association

How the landing page design can affect conversion

Experiment ID: #1365

Texas State Historical Association

Experiment Summary

Timeframe: 06/24/2015 - 09/25/2015

A recent experiment was done to bring older landing pages up to the latest levels of optimization. However, what this experiment proved is that a design that works for one page may have the adverse reaction on another. Based upon these results, we decided to revalidate the main Heritage donation page to ensure that we were using a design that was achieving the best possible results for us.

Research Question

Which design will allow for the greatest number of gifts and revenue?


C: Secured Template
T1: Join Heritage Template


 Treatment NameConv. RateRelative DifferenceConfidence
C: Secured Template 38.3%
T1: Join Heritage Template 31.9%-16.7% 76.9%

This experiment has a required sample size of 429 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

After 90 days, this test had not yet reached statistical significance so we decided to turn it off in order to move to the next experiment. However, based upon a level of confidence above 85%, we can classify this as “approaching validity.” This result reinforces that the current simple design of the main donation page is effective with the current amount of copy.

Experiment Documented by NextAfter

Question about experiment #1365

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.