How will the presence of a value proposition banner in checkout affect conversion? Experiment ID: #15988

Compassion International

Experiment Summary

Ended On: 5/21/2019

Compassion International has a three-step checkout process to sponsor a child. They have long known that there is a drop-off rate for each step of the process, and have wanted to optimize that—knowing that people who have already selected a child have a high motivation and should be more likely to complete their transaction. One thing they noticed is that once a donor entered the shopping cart, there wasn’t much value proposition copy to motivate them through the transaction. They had seen tests that showed that a value proposition “sticky banner” in the process increased conversion rates for donations—even on high-conversion-rate pages. They planned to test different value proposition messages, but also wanted to aggregate the data to determine if the presence of a banner with any message increased conversion. 

Research Question

Will a “sticky banner” with any value proposition outperform a control with no banner?


C: Control
T1: Treatment #1


Treatment Name Conv. Rate Relative Difference Confidence
C: Control 36.0%
T1: Treatment #1 41.0% 13.7% 95.5%

This experiment has a required sample size of 760 in order to be valid. Since the experiment had a total sample size of 2,654, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 13.7% increase in conversion rate
× 0% increase in average gift

Key Learnings

A banner with any message increased conversion by 13.7%. Most of these increases were realized on desktop, though mobile had no decreases and no banner message had less than a 10% lift. This confirmed Compassion’s hypothesis that adding value proposition to the checkout process could increase motivation to complete the three-step process. Next, they looked at a different dataset to determine which messages drove the highest increases. 

Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.