How concisely communicating the value proposition impacted conversions on a main donation page
The Gospel Coalition
Experiment Summary
Timeframe: 03/15/2022 - 04/11/2022
We previously tested the introduction of value proposition on TGC’s main donation page and saw that it was directionally decreasing conversion on the page so we ended the test and went back to the drawing board for new ideas on how to improve conversion and achieve the lift we were looking for. We still believed that some value proposition would be better than no value proposition so we wanted to condense the copy and see if a shorter presentation of the value prop would get greater results. We developed a treatment that acknowledged the visitor’s interest in making a gift in the headline, highlighted the need at hand, and quickly outlined the direct impact the donor’s gift would make. We also removed the header and footer with outbound or distracting links to keep the visitor focused on the donation page ask and make the desired call to action clear. Traffic was split 50/50 to the page.
Research Question
We believe that concisely communicating the value proposition for main donation page visitors will achieve an increase in donor conversion.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Main Donation Page - Control | 4.1% | |||
| T1: | New Headline + Body Copy - Treatment | 8.9% | 116.4% | 99.3% |
This experiment has a required sample size of 204 in order to be valid. Since the experiment had a total sample size of 773, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 116.4% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running the experiment for 3 weeks, we achieved a 116.4% increase in conversion rate with 99% level of confidence so the results are valid. We saw valid lifts for desktop visitors (130.8% increase), returning visitors (295.2% increase), and email visitors (866.7%) all with 100% level of confidence. We believe this happened because we acknowledged their reason for visiting the page and began the conversation to communicate the impact of making a gift clearly and succinctly to page visitors. One thing to keep in mind is that revenue experienced a 56.1% decrease with an 87% level of confidence due to a few large gifts on the control version that are outliers impacting these results but we should also consider if there is anything on the treatment page that could be causing donors to choose a lower average gift. To combat the indication that there is a directional decline in revenue, on a future iteration of a test for this page, we could highlight a “most popular” gift choice on the array to help improve average gift. We could also call more attention to recurring giving to increase the overall value of a donor upon conversion.