Dallas Theological Seminary

How personalization on a donation page impacted donations and revenue

Experiment ID: #40446

Dallas Theological Seminary

The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.

Experiment Summary

Timeframe: 09/08/2020 - 10/02/2020

As part of a giving day campaign, DTS was offering a content offer and wanted to test using the donor’s first name in the headline to see if this would drive conversion rate on the instant donation page.

Research Question

Will personalization on an instant donation page after a content offer increase donor conversion?

Design

C: Control
T1: First Name Personalization

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 3.6%
T1: First Name Personalization 3.2%-12.2% 58.1%

This experiment has a required sample size of 13,056 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

While we did not reach validity with this experiment, when evaluating the results and drilling down even further by each segment, we noticed that while overall conversion rate decreased on the page, we saw an 89.4% lift in average gift—specifically with the returning visitors segment—with an 88% level of confidence. Across all traffic we saw a 40.8% lift in average gift with an 82% level of confidence. These findings in the directional lift in average gift will be used for future testing and will be useful for developing what language we present on donation pages to returning visitors moving forward.


Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #40446

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.