How shortening copy on mobile acquisition page impacts conversion
Care Net
Experiment Summary
Ended On: 03/23/2018
Care Net offers a Pro-life Pledge in front of their online course. A significant amount of their traffic for this offer comes from mobile. Even though they were able to acquire names at a healthy cost, they hypothesized that they might be able to improve upon that cost by reducing the amount of copy on the mobile experience. They set up an experiment to only target mobile and removed course testimonials and any copy that was focused on anything but the value of the course.
Research Question
Would shortening the copy on the mobile version increase acquisition?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 29.1% | ||
| T1: | Shortened Value Prop | 32.8% | 12.9% | 96.8% |
This experiment has a required sample size of 1,168 in order to be valid. Since the experiment had a total sample size of 2,802, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 12.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
By shortening the copy on the treatment version, we were able to increase name acquisition by 13%! This lift reinforces the idea that longer copy on mobile creates friction in the acquisition process.