How adding communal language on a sticky bar impacted website clicks
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Ended On: 03/01/2021
When launching a new course, DTS adds a sticky bar to their website to allow visitors to access the course. The standard language is about getting access to the newest free online course, but we wanted to test how addressing website visitors conversationally like friends would affect the clickthrough rate on the sticky bar.
Research Question
We believe that adding communal language for online course takers will achieve an increase in website visitors.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.3% | ||
| T1: | Treatment #1 | 1.7% | 29.2% | 99.9% |
This experiment has a required sample size of 7,894 in order to be valid. Since the experiment had a total sample size of 43,839, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
29.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
What we found was that making the language more communal and less transactional provided a 29.2% increase in traffic to the course page. Letting people know that they were invited to take the course because of their existing relationship with the organization was more effective than simply giving them a highly visible point of access. Establishing these findings will be helpful for building language to incorporate in future course offerings. Since we validated the experiment for website traffic, we can do further testing to see if this language is similarly effective in other channels like Facebook ads and email copy.