How reducing the number of membership levels affects conversion
Texas State Historical Association
Experiment Summary
Timeframe: 12/18/2015 - 01/04/2016
During their calendar year end campaign, the Texas State Historical Association created a landing page dedicated to acquiring new members. Their membership program consists of 6 different levels ranging from $25 up to $1,000. After analyzing previous year end campaigns, we realized that the number of new members that select gifts above $250 are few and far between.
Since few people select them, we had the hypothesis that these upper membership levels may actually be decreasing conversion due to the amount of mental friction that they cause. We decided to test a form that only had four membership levels to see if we could improve the overall conversion of the page.
Research Question
Which gift array will result in the greatest number of new members?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | 6 Options | 5.5% | $50.95 | ||
| T1: | 4 Options | 4.6% | -17.1% | 90.0% | $53.02 |
This experiment has a required sample size of 4,138 in order to be valid. Since the experiment had a total sample size of 5,820, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
The experiment revealed that the donation form with four levels of membership actually decreased conversion by 17.1%. Our hypothesis is that these additional membership levels actually provided clarity around the benefits of each level which were mentioned in the text above the form. Knowing which levels offered which benefits allowed the new members to feel safer in their selection of membership.