How a radical redesign affects the force of the value proposition Experiment ID: #3793

Texas State Historical Association

Experiment Summary

Timeframe: 9/1/2013 - 9/30/2013

The Texas State Historical Association had several initiatives designed to grow their membership base. However, the membership page presented tremendous friction. The experience used a shopping cart, though it was asking for donations. It also presented four different membership options, but did not give distinction between them. And the copy on the page was not very specific, and made unclear and unsubstantiated claims.

The team created a radical redesign with longform copy to appropriately convey the value proposition. They added supporting copy that highlighted exclusivity and credibility, and gave a full explanation of why the visitor should join the TSHA as a member. This page also included evidentials to back up each claim. Finally, the membership levels were quantified and given distinct benefits, with one preferred level preselected.

An A/B test was then launched to determine a winner.

Research Question

Will a radical redesign that reduces friction and dramatically increases the force of the value proposition increase donor conversion?

Design

C: Old Membership Experience
T1: New Membership Page

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Old Membership Experience 1.1%
T1: New Membership Page 2.6% 146.5% 100.0%

This experiment has a required sample size of 700 in order to be valid. Since the experiment had a total sample size of 4,312, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 146.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

This experiment showed how affecting the force of the value proposition can radically affect donor conversion. Though the underlying value proposition of the TSHA remained the same, the information was presented in a way that caused many more visitors to understand it, believe it, and want to join.

This experiment also revealed that TSHA members are readers, which makes complete sense given their affinity for history. Long form copy gives them more information, refuting the commonly held conventional wisdom that “people don’t read on the internet”.

 


Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.