Hillsdale College

How including an image influences conversion rates

Experiment ID: #18618

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 08/18/2019 - 09/17/2019

For Hillsdale’s free course offer, we wanted to test increasing clarity through providing an image related to the course subject that was used to promote the offer on the email acquisition page. The control version used the same copy and contained no image. Our hypothesis was that adding clarity and familiarizing the audience with the author of the works covered in the course content would increase the perceived value of the offer. We wanted to see if showing a black and white illustrated image would act as a distraction and introduce friction or improve conversion.

Research Question

Will including a related image on the course registration page improve email acquisition rate for the Jane Austen online course?

Design

C: treatment
T1: treatment -- w/ image

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: treatment 5.6%
T1: treatment -- w/ image 6.4%13.9% 68.2%

This experiment has a required sample size of 7,173 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

After running for a month, we were not able to validate the results of the experiment and will need to pursue further testing with this hypothesis. 

Adding some clarity to the offer—enough to remind the visitors of the topic that would be covered in the course and increase the perceived value of the offer—without acting as a distraction, did result in additional course registrations on the treatment page,


Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #18618

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.