How adding a pledge to the front of an online course impacts conversion rate
Care Net
Experiment Summary
Timeframe: 03/09/2018 - 03/14/2018
Care Net offers a free online course called, Pro-life 101. They have been testing and optimizing this campaign since it started last summer. Through this testing, they have been able to cut down the cost of acquiring a new course sign-up in half. Since then, they have done additional testing but have not been able to impact acquisition cost any more. We proposed a radical experiment for the course that would add a pro-life pledge to the beginning of the course in an effort to decrease the initial cost to get people to the course sign-up page. Once on the sign-up page, we also removed additional form fields that weren’t needed and passed through a person’s email address from the pledge page.
Research Question
Would adding a pledge to the front of the online course increase conversion of course sign-ups?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.38% | ||
| T1: | Treatment #1 | 1.0% | 172.6% | 100.0% |
This experiment has a required sample size of 1,194 in order to be valid. Since the experiment had a total sample size of 70,314, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 172.6% increase in conversion rate
× 0% increase in average gift
Key Learnings
By putting a pledge in front of the online course, not only were we able to acquire names initially for much lower ($0.42 a name) through the pledge, we were able to increase course sign-ups by 172%! In addition, we were also able to decrease acquisition cost for the course significantly by 66%. This significant improvement and lift can be attributed to a few things:
1. The lower perceived cost of signing a pledge attracted more people to this offer resulting in more people seeing and reading the value prop for the course.
2. The pledge acted as a micro-yes in the conversion funnel leading them to the online course as the next logical step
3. By removing the additional form fields for the online course and pushing a person’s email address through from the pledge, we reduced significant friction in
the sign-up process.
Since this experiment, all traffic has been pushed towards this approach to get more people signed up for the online course and we will continue to test and optimize this new course experience.