How placement of a sticky bar affects email acquisition on mobile devices Experiment ID: #9694

Leadership Institute

Experiment Summary

Ended On: 11/15/2018

Leadership Institute sought to acquire as many email addresses as possible from their flagship news site, CampusReform.org. They had put email acquisition offers on the site through “sticky bars” and sought to optimize both through testing and optimization on both desktop and mobile. They had tested to a sticky bar that appeared when someone scrolled through 25% of the page at the bottom of the page. They hypothesized that this might be interrupting the user experience too much on mobile, as the content removed their ability to scroll down. They decided to test placement—from the current “bottom anchored” bar to the “top anchored” bar.

Research Question

How will a sticky bar anchored to the top of the mobile experience perform against one anchored to the bottom?

Design

C: Lower Sticky Bar
T1: Upper Sticky Bar

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Lower Sticky Bar 2.3%
T1: Upper Sticky Bar 4.9% 110.3% 96.2%

This experiment has a required sample size of 434 in order to be valid. Since the experiment had a total sample size of 873, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 110.3% increase in conversion rate
× 0% increase in average gift

Key Learnings

The top-anchored sticky bar produced a 110% lift in name acquisition than the bottom-anchored sticky bar—which showed that where you place the sticky bar in terms of user experience has a tremendous impact on the results it produces.

Ultimately, this test established a new standard—future testing should focus on design and value proposition, but this provides a helpful framework for understanding user experience.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.