How the way a content offer is presented at the end of an article impacts name conversion Experiment ID: #21425

Focus on the Family

Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.

Experiment Summary

Timeframe: 3/9/2020 - 3/20/2020

On the Focus on the Family website, there is a content offer at the end of each article to people to sign-up for a free video series. The video series is relevant to the content they are reading, uses a picture to show them who the teacher of the video series is, communicates what the video series will help them with, and has a button where people can be taken to a sign-up page to get the offer. This had been running on the site for a while and we hypothesized that we might be able to get more sign-ups by presenting the offer as a sticky bar at the bottom of the page instead of an inline article offer.

Research Question

Which placement method would acquire more names?

Design

C: In-line article offer
T1: Sticky bar at bottom of article

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: In-line article offer 0.50%
T1: Sticky bar at bottom of article 0.16% -68.3% 100.0%

This experiment has a required sample size of 2,158 in order to be valid. Since the experiment had a total sample size of 76,765, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 68.3% decrease in conversion rate
× 0% increase in average gift

Key Learnings

While the sticky bar had greater presence on the page since it was “stuck” to the bottom of it, it saw significantly less conversions. In fact, the stick bar decreased name conversion by 68%. The tactic behind the sticky bar seemed to make practical sense, but it failed to be able to communicate the much needed value proposition of the offer that the in-line offer was able to do. The inline offer took a greater amount of real estate allowing it to fully communicate the benefit of the offer and serve as a logical next step after a person finished reading the article content.


Experiment Documented by...

Courtney Gaines

Courtney is a Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.