How adding a testimonial affects email sign ups on a landing page Experiment ID: #21192

Back to the Bible

Experiment Summary

Ended On: 5/29/2020

Back to the Bible launched a new eBook 7¬†Secrets of Worry-Free Living and wanted to test whether including a review from the leader of a Christian men’s conference, Brian Doyle, would increase email sign ups on the landing page. All other landing page elements were identical.

Research Question

Will utilizing a review on a landing page increase email acquisition?

Design

C: Control - no Testimonial
T1: Treatment - w/ Testimonial

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control - no Testimonial 48.6%
T1: Treatment - w/ Testimonial 44.7% -7.9% 94.8%

This experiment has a required sample size of 1,296 in order to be valid. Since the experiment had a total sample size of 2,541, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

While we turned off the experiment before reaching a valid level of confidence, we were seeing an overall decline in conversion rate on the treatment. Future experiments will test using other various forms of social proof such as number of eBook downloads or a testimonial from a peer.


Experiment Documented by...

Rebekah Josefy

Rebekah is Optimization Associate. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.