How aligning the value proposition with the audience affects clickthrough rate Experiment ID: #9445
Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.
Ended On: 9/19/2018
Hillsdale College was running a campaign to fund their new online course. They were trying to get traffic to the donation page from several of their properties that had inbound traffic. They used sticky bar banners to attempt to motivate visitors to give. They tested for different motivations: FirstThey wanted to ask users to help launch the course by asking a question.Then they tried a second treatment that was designed to provoke curiosity using “find out how “language. Third, they tried a command: “Help launch Hillsdale’s new Congress course”, and finally, they tried a more complex value proposition: “Help educate America for liberty with Hillsdale’s new course.” They launched the sticky bar banners as a four-way test to determine a winner on the Hillsdale Dialogues page, which attracts an audience of intellectually driven visitors who arrive to listen to a podcast.
Which value proposition will deliver the most clicks?
|Treatment Name||Click Rate||Relative Difference||Confidence|
This experiment has a required sample size of 538 in order to be valid. Since the experiment had a total sample size of 4,338, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
169.5% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
The fourth value proposition, “Help educate America for liberty with Hillsdale’s new online course” delivered almost twice as many clicks as any of the other banners. This show that the edgy intellectually-driven audience responded well to a more complex value proposition than just a simple ask. It showed that they buy into the mission behind the course—not just the simple launch of a new course. Typically this would be an interesting, but not insightful learning, except for the fact that it was contradicted by another test on Hillsdale’s online courses page in which the “launch” language won. This shows the importance of testing across multiple properties with different messages so that you can match the right message to the right audience.