How reducing friction and bringing the problem to the headline affect conversion rate on desktop devices
Prison Fellowship International
Experiment Summary
Timeframe: 08/26/2021 - 10/14/2021
PFI had been running ads promoting child sponsorship for many months, and had seen a predictably low conversion rate, as a monthly ask of $39 is a high bar for many donors.
Wanting to improve conversion on their sponsorship pages, they tested two changes at once. First they looked at the headline, which appeared somewhat vague. They wondered if presenting the problem in the headline—that the child was in prison—would increase motivation to step in and help through sponsorship.
Secondly, when the sponsor was ready to commit, they had two options: they could either scroll all the way down the page past a large amount of copy and a “view more children” button, or hit a sponsor now button that, on desktop, was all the way to the right of the page and would scroll them to the bottom. Once arriving at the bottom of the page, there was a lack of orientation as to what to do.
They decided to test bringing the donation form all the way up to the top of the page, under the copy, so that it was right in front of the user as they made the decision to give. They also linked the copy to the form through a call-to-action headline.
They made both of these changes at once and launched an experiment to discover the impact on conversion.
Research Question
We believe that bringing the problem into the headline and streamlining the donation experience for child page visitors will achieve an increase in conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.2% | ||
| T1: | Treatment #1 | 3.5% | 199.1% | 95.7% |
This experiment has a required sample size of 322 in order to be valid. Since the experiment had a total sample size of 683, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 199.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
At first the results of the test looked flat, or trending slightly downward…
However, splitting the traffic into desktop and mobile revealed something fascinating. The treatment produced a 199% valid lift in conversion for desktop devices, which make up 39% of conversions.…which is being offset by a 51% near-valid decrease in conversion for mobile devices, which make up 61% of conversions.