How using an inline form for a particular kind of offer on the homepage impacts name conversion.
Care Net
Experiment Summary
Timeframe: 04/11/2018 - 05/12/2018
On the Care Net homepage, they offer people the opportunity to sign a pro-life pledge. In a previous experiment where they offered people a free online course, we found that a button increases conversion over an inline form. As we thought through the kind of offer they were now giving people – a pledge to sign instead of an online course, we hypothesized that the button approach might not be the best method. To test this, we developed a treatment that added an inline form to the offer on the page. No other content or copy changed.
Research Question
Would an inline form on the homepage increase conversion for the pro-life pledge?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 5.1% | ||
| T1: | Pledge Form | 6.7% | 31.7% | 99.0% |
This experiment has a required sample size of 1,639 in order to be valid. Since the experiment had a total sample size of 5,673, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 31.7% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment increased name acquisition by 32%. The results from this experiment are opposite of the results from a similar experiment for the pro-life online course. Why is that? Because the pro-life course had a much higher mental cost and commitment. People needed more information about the online course before they were ready to sign up for it. That is why a button instead of an inline offer performed better for the course. The pro-life pledge requires a much lower commitment. People will either align or not align with the position of a pledge.
What we can take away from this experiment is that we need to be mindful of the kind of offer we have to determine the acquisition call-to-action method.