Texas State Historical Association

How changing the website navigation to address multiple segments affects traffic

Experiment ID: #2325

Texas State Historical Association

Experiment Summary

Timeframe: 09/29/2015 - 10/22/2015

On the Texas State Historical Association’s website, they had a navigation option dedicated to member acquisition. The link was found toward the end of the navigation and directed traffic to a page with more information on the benefits of membership.

We had a hypothesis that this one button and subsequent page did not sufficiently address the needs of the various segments. In this situation, we had both potential members and existing members visiting the website. We decided to test out a new navigation with two options that would address their distinct needs.

Research Question

Which website navigation will drive more traffic to the giving pages?

Design

C: Membership
T1: Join TSHA / Renew Membership

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Membership 0.19%
T1: Join TSHA / Renew Membership 0.22%15.6% 95.2%

This experiment has a required sample size of 175,731 in order to be valid. Since the experiment had a total sample size of 356,454, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    15.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

By splitting the Membership navigation into two options, we were able to directly address the two distinct segments (existing and potential members).  This change led to a 15.6% increase in the number of visitors to the giving pages.

There was also a secondary benefit of the two navigation options. We were able to create two distinct pages that were better able to address the needs of the two different segments with targeted value propositions and calls to action.


Experiment Documented by NextAfter

Question about experiment #2325

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.