Dallas Theological Seminary

Understanding the perceived value of video to email recipients

Experiment ID: #407

Dallas Theological Seminary

The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.

Experiment Summary

Timeframe: 12/28/2014 - 12/30/2014

Video is often used as a way to increase visitor engagement on donation landing pages. However, it was our hypothesis that the videos actual distracted from the ultimate goal, giving. As a result, we created an experiment to test a video against a text version of the same message.

Research Question

Will using text instead of a video will allow donors to focus on the call-to-action which will increase the response rate of the campaign?

Design

C: Video Email
T1: Text Email

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Video Email 6.7%
T1: Text Email 6.5%-2.9% 39.4%

This experiment has a required sample size of 121,937 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

With no significant difference between click rates and nearly identical response rates (both are 0.1%), the effort put into the video may not be necessary this late in a year end campaign.


Experiment Documented by NextAfter

Question about experiment #407

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.