How adding a pledge to the front of an online course impacts conversion rate Experiment ID: #8665

Care Net

Experiment Summary

Timeframe: 3/9/2018 - 3/14/2018

Care Net offers a free online course called, Pro-life 101. They have been testing and optimizing this campaign since it started last summer. Through this testing, they have been able to cut down the cost of acquiring a new course sign-up in half. Since then, they have done additional testing but have not been able to impact acquisition cost any more. We proposed a radical experiment for the course that would add a pro-life pledge to the beginning of the course in an effort to decrease the initial cost to get people to the course sign-up page. Once on the sign-up page, we also removed additional form fields that weren’t needed and passed through a person’s email address from the pledge page.

Research Question

Would adding a pledge to the front of the online course increase conversion of course sign-ups?

Design

C: Control
T1: Statement of Belief

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.38%
T1: Statement of Belief 1.0% 172.6% 100.0%

This experiment has a required sample size of 1,194 in order to be valid. Since the experiment had a total sample size of 70,314, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 172.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

By putting a pledge in front of the online course, not only were we able to acquire names initially for much lower ($0.42 a name) through the pledge, we were able to increase course sign-ups by 172%! In addition, we were also able to decrease acquisition cost for the course significantly by 66%. This significant improvement and lift can be attributed to a few things:

  1. The lower perceived cost of signing a pledge attracted more people to this offer resulting in more people seeing and reading the value prop for the course.
  2. The pledge acted as a micro-yes in the conversion funnel leading them to the online course as the next logical step
  3. By removing the additional form fields for the online course and pushing a person’s email address through from the pledge, we reduced significant friction in the sign-up process.

Since this experiment, all traffic has been pushed towards this approach to get more people signed up for the online course and we will continue to test and optimize this new course experience.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Courtney Gaines

Courtney is the Senior Director of Optimization at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.