How improving clarity with more specific language in a header menu option affects click-through rates
Save the Children
Experiment Summary
Timeframe: 08/30/2022 - 09/15/2022
We changed the copy of the Team Tomorrow navigation header on the homepage to see what has the highest click-through-rate
Research Question
We believe more specific navigation language will increase click-through rates by adding clarity around the monthly giving option on a navigation header.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control (Give with Impact) | 0.31% | ||
| T1: | Monthly Support | 0.81% | 157.1% | 99.9% |
| T2: | Ongoing Support | 0.53% | 69.7% | 91.1% |
This experiment has a required sample size of 2,583 in order to be valid. Since the experiment had a total sample size of 15,096, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
157.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
Notable Results & Recommendations:
• Increased unique clicks for treatment #1 (b) (+157%) at a statistically valid level of confidence (99.9% LoC).
• Using support language rather than give language on the homepage navigation increases the click-through-rate significantly.
• Directionally, donor conversion was slightly higher for both treatments.
• Our hypothesis that using more supportive language would yield higher clicks was confirmed and demonstrated a 157.1% increase in clicks for All Traffic with a 99.9% statistical level of confidence.
• Recommendation is to replace “Give with Impact” with “Monthly Support.”