Marriage Today

LWF – Email Subject Line Test

Experiment ID: #54697

Marriage Today

Experiment Summary

Ended On: 03/15/2021

Testing subject lines and the preheader. The results on opens and clicks were very similar, however the treatment had a higher conversion rate when it came to donations. While not above the 95% confidence level it does give us something to build upon.

Research Question

Which subject line will produce the best results for LWF?


C: Control
T1: Treatment #1


 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.04%
T1: Treatment #1 0.10%150.0% 74.3%

This experiment has a required sample size of 15,646 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

The subject line that was tested may have done a better job of attracting people that wanted to get the series as it framed the content as what a person would get out of the book when they read it.

Experiment Documented by NextAfter

Question about experiment #54697

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.