How a radically redesigned donation page for monthly partnership impacted donations.
Texas Public Policy Foundation
Experiment Summary
Timeframe: 11/03/2022 - 12/05/2022
A public policy client has a special membership club for monthly donors. Someone can join this club when making a monthly donation of at least $10, and from there they’ll receive special perks and have an elevated impact on the organization and its cause. The donation page where someone can make a monthly gift to join the club is resulting in a low instant conversion rate. Considering the amount of traffic that reaches the page, and after brainstorming several potential ways to improve conversion, we’ve opted for a radical redesign test to determine its impact on the instant donor conversion rate.
We will test these elements:
- Expanding the value proposition of the headline
- Moving the inline quote outside of the main copy and into a quote box on the side
- Calling out the specific impact of making a monthly gift via bullet points
- Moving the (501(c)3) tax language to the bottom of the page below the giving button
- Setting the default gift array amount to $10
- Changing “Donor Information” to “Your Information” in the giving form
- Removing an email subscription opportunity from above the giving button
- Removing only on mobile the social proof slide in informing the page visitor that [someone] just joined the membership club
We want to determine if this combination of changes will improve the page/donation experience for the visitor, provide additional clarity on the benefits of joining the membership club, and ultimately increase donor conversions.
Research Question
We believe that conducting a radical redesign on a monthly donor membership donation page for potential donors will achieve an increase in instant donor conversion rate because potential donors will gain clarity on the benefits of joining the membership club and will have a cleaner, more user-friendly page engagement.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 1.9% | $0.00 | ||
| T1: | Redesign | 0.46% | -76.5% | 95.2% | $0.00 |
This experiment has a required sample size of 404 in order to be valid. Since the experiment had a total sample size of 848, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 76.5% decrease in conversion rate
× 0% increase in average gift
Key Learnings
The results are in. The treatment (radical redesign) showed a valid 76.5% DECREASE in instant donor conversion rate. The control variant produced 8 conversions to the treatment’s 2.
Our swing for the fences resulted in a whiff. The results were negative across segments. Unfortunately, a radical redesign cannot tell us how each change influenced the user experience and donor conversion rate overall. To simplify this test and make more sense of the results, our next test should isolate one or two variants. We may want to go down the list of changes so we can truly determine which change(s) turned potential donors off.