Luther Seminary

How Adding Value Proposition Language near the ‘Give’ Button on a Homepage Affects Conversion Rates

Experiment ID: #72382

Luther Seminary

Experiment Summary

Timeframe: 09/27/2021 - 12/29/2021

Although Luther Seminary has a large, bright donation callout on their homepage, we observed that it was not driving the expected level of donations.

Since there were other bright design elements competing for attention around the donation callout, and because Luther Seminary’s homepage is somewhat ‘busy’, with many navigation choices for visitors, we hypothesized that perhaps visitors weren’t being given enough of a reason to choose to donate, with so many other options from which to choose.

Previous experiments have shown that clearly communicating a value proposition motivates donors to give. In light of these past learnings, we decided to test how adding value proposition language near the ‘GIVE’ button on Luther Seminary’s homepage might improve conversion rates without necessitating a redesign of their homepage.

Research Question

We believe that adding value proposition language near the give button on a homepage for web traffic will achieve higher conversion rates.

Design

C: Control

T1: Treatment

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence	Average Gift
C:	Control	2.6%			$0.00
T1:	Treatment	0.91%	-65.2%	100.0%	$0.00

This experiment has a required sample size of 463 in order to be valid. Since the experiment had a total sample size of 4,246, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

0% increase in traffic
× 65.2% decrease in conversion rate
× 0% increase in average gift

Key Learnings

After running our experiment for 90 days, we observed a 21% decrease in conversion rate with an 82.5% level of confidence.

While this result was surprising, it does provide context and points the direction for future optimization.

Most interestingly, while returning visitors showed a 65% decrease in conversion rates, the new treatment did result in a 127% increase in donations from new traffic (including an impressive 295% lift in paid traffic donations).

This provides some interesting insights:

first, we can hypothesize that for first time visitors, especially those being acquired via paid traffic, the value proposition was helpful in inspiring them to give

second, there could be a variety of factors leading to the decrease in donations from returning traffic, one of which may be the sheer number of things competing for a visitor's attention on the homepage. A quick heuristic analysis of the homepage indicates that it is quite busy and may cause friction for returning visitors. However, an analysis of website data (such as page path, exit, and bounce rate reports), combined with scroll tracking / heat mapping, should be conducted to see what people are doing when they are landing on the homepage and where they are going when they click through

It is doubtful that the inclusion of helpful value prop language was the key motivating factor in reducing donations from returning visitors, but additional analysis would be required to determine this with a high level of certainty.

Experiment Documented by Jeff Giddens

Jeff Giddens is President of NextAfter.

Question about experiment #72382

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.