Luther Seminary

How removing thank you from the header on the instant donation page affects donor conversion rate

Experiment ID: #112594

Luther Seminary

Experiment Summary

Timeframe: 10/31/2022 - 11/11/2022

Luther Seminary is offering an online course to study the Book of Revelation. We had previously seen other organizations test the concept of removing “thank you” from the header of the donation page as that typically ends a user’s thought process when signing up for something or completing a purchase. Usually, when you see the thank you confirmation page, you think to yourself – Okay, I’m done now, so I’m leaving the page.

We predicted for this online course that users are leaving the donation page after seeing “thank you” at the very top, so we want to test changing that language so something that lets the user know that they are not done yet.

Research Question

We believe that removing thank you from the header on the donation page for course enrollees will achieve a higher donor conversion rate.

Design

C: Control
T1: Treatment

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 3.4%$0.00
T1: Treatment 7.8%132.0% 96.0%

This experiment has a required sample size of 202 in order to be valid. Since the experiment had a total sample size of 456, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 132.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment page saw a 132% increase in donor conversion rate with a 96% level of confidence. We believe by saying “But before you go, will you consider something?” that the user’s thought process is no longer interrupted therefore they are more likely to stay on the page to read the value proposition rather than bouncing as soon as they see “thank you.”

Notable changes to other metrics due to the experiment:

  • Revenue experienced a 140.2% increase with a 97% level of confidence.

We believe revenue experienced such a high lift on the treatment page because we were able to keep double the amount of people on the page. After staying and reading the copy on why they should give, they were more likely to make a donation.


Experiment Documented by NextAfter

Question about experiment #112594

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.