Luther Seminary

How radically redesigning a landing page with clarifying copy and removing navigation effects email acquisition rate

Experiment ID: #97622

Luther Seminary

Experiment Summary

Timeframe: 06/16/2022 - 08/04/2022

Luther Seminary had been running their acquisition pages with navigation in the header and footer of the page. They wanted to test what would happen if we removed the navigation from the page to see if this would increase email acquisition.

We tested this concept on their Cosmic Adventure online course landing page. On top of this page having the navigation bars, the offer was not performing well for a consistent amount of time, so they also wanted to test changing the headline and body copy of the page to see if providing clarity for the user as to what they would receive in this course would help increase email acquisition.

The original page language was vague throughout the headline, sub-headline, and body copy. Within the new copy, we wanted to be very specific in the headline and subheadline by stating exactly what this course would do for them – give them answers to questions between science and faith through an informative course. Within the body copy, we wanted to focus on curiosity and clarity.

Research Question

We believe that removing the navigation and providing clarity in the page copy for users signing up for this online course will achieve a higher email acquisition rate.


C: Control
T1: Same copy, no nav/footer
T2: New copy, no nav/footer


 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 37.0%$0.00
T1: Same copy, no nav/footer 40.8%10.3% 73.6%$0.00
T2: New copy, no nav/footer 44.7%20.8% 96.7%$0.00

This experiment has a required sample size of 470 in order to be valid. Since the experiment had a total sample size of 1,140, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 20.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

This experiment (treatment page 2) produced a valid increase in the email sign-up rate of 44.7% with a 96.6 level of confidence. In looking at all three variants, we believe that removing the navigation in the header and footer provided the biggest impact on our sign-up rate. On top of that, changing the page copy to be more clarifying for the user as to what this course will give them ultimately provided us with the valid increase needed.

It is recommended that this be tested on other acquisition pages separately, so we can see the true impact of removing navigation vs adding clarifying copy.

Notable changes to other metrics due to the experiment:

  • Revenue experienced a 113.8% increase with a 86% level of confidence.

Experiment Documented by NextAfter

Question about experiment #97622

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.