How reducing friction on mobile devices affects name conversion rate
Human Liberty
Human Liberty is an international network of non-profit organizations working to protect and promote human liberty worldwide.
Experiment Summary
Timeframe: 06/12/2014 - 07/23/2015
Human Liberty was promoting a petition directed at the International Justice system in support of North Koreans held in captivity by a totalitarian government. They noticed that much of their high-converting traffic was coming from mobile devices. Conversion was relatively low, even for the highly-targeted audiences they focused their advertising on. So, they proposed a test to discover exactly how highly motivated the visitors were. They created a new treatment with drastically shorter copy. If the audience was already highly motivated, the additional copy on the control might actually be slowing them down on a mobile device.
Research Question
Does reducing copy on mobile devices increase the likelihood of conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control - Long copy | 15.3% | ||
| T1: | Treatment - Shorter Copy | 26.9% | 76.2% | 100.0% |
This experiment has a required sample size of 105 in order to be valid. Since the experiment had a total sample size of 4,694, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 76.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
The new treatment increased conversion by 76.2% — validating the hypothesis that mobile traffic was not only highly motivated and needed less “convincing”, but also responded well to shorter messages that took less time to read. This was highly valuable for all of Human Liberty’s mobile-focused campaigns.