How a time-sensitive value proposition affects conversion rate for donors Experiment ID: #4998

Dallas Theological Seminary

The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.

Experiment Summary

Timeframe: 9/9/2016 - 9/20/2016

Dallas Theological Seminary was running a fundraising campaign for North Texas Giving Day.They had previously tested into a value proposition centered around their mission: growing the kingdom. However, they wanted to test to see if a very time-sensitive specific matching challenge ask would be a greater motivator for donors and nondonors. Since the two segments had different levels of affinity and motivation, they wanted to test them separately. They created a treatment page and showed it to both donors and nondonors independently, with a 50-50 split to try to determine which value proposition was a greater motivator for this specific campaign.

Research Question

Will a matching challenge ask outperform a general value proposition ask during a time sensitive campaign for a donor segment?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: Teach Truth
T1: Matching Challenge

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Teach Truth 8.5%
T1: Matching Challenge 6.5% -23.7% 68.0%

This experiment has a required sample size of 1,296 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

The treatment page produced a 23% decrease in donor conversion rate although this, like the nondonor test, was not statistically valid due to low traffic for this campaign. Because it was not statistically valid, these numbers cannot be interpreted to infer that the new value proposition was less motivating to donors. This test was closer to validity than the nondonor test, but even when combining the data from the two tests, there was not a clear winner or statistical validity.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.