Dallas Theological Seminary

How communal language impacted donation page visitors at year-end

Experiment ID: #46792

Dallas Theological Seminary

The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.

Experiment Summary

Ended On: 01/28/2021

DTS had a sticky bar running on their site at year-end with language about a deadline to reach a $500,000 Challenge Gift. We wanted to test a hypothesis that adding softer, more relationship-based language in place of the transactional, deadline focused language would drive additional traffic to the donation page.

Research Question

We believe that using relationship-based language for site visitors will achieve greater donation page traffic.


C: Control
T1: Treatment #1


 Treatment NameClick RateRelative DifferenceConfidence
C: Control 0.60%
T1: Treatment #1 0.24%-60.0% 96.5%

This experiment has a required sample size of 2,461 in order to be valid. Since the experiment had a total sample size of 5,746, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    60.0% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

What we found after running the experiment was the treatment copy decreased conversions by 60%. For future experiments, we would want to match the value proposition more closely on the treatment in order to truly isolate the deadline and transactional Challenge Gift language from the communal language. For example, a gift to “reach lives with the truth of the Bible” could have been perceived as more motivating than “reach the world with God’s Word” so the results are too convoluted to draw a true conclusion. We would also like to run a similar test during a different season to see if this alters results at all since we know that donors are typically highly motivated to give during the month of December.

Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #46792

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.