Prison Fellowship International

How reducing friction and bringing the problem to the headline affect conversion rate on desktop devices

Experiment ID: #78943

Prison Fellowship International

Experiment Summary

Timeframe: 08/26/2021 - 10/14/2021

PFI had been running ads promoting child sponsorship for many months, and had seen a predictably low conversion rate, as a monthly ask of $39 is a high bar for many donors.

Wanting to improve conversion on their sponsorship pages, they tested two changes at once. First they looked at the headline, which appeared somewhat vague. They wondered if presenting the problem in the headline—that the child was in prison—would increase motivation to step in and help through sponsorship.

Secondly, when the sponsor was ready to commit, they had two options: they could either scroll all the way down the page past a large amount of copy and a “view more children” button, or hit a sponsor now button that, on desktop, was all the way to the right of the page and would scroll them to the bottom. Once arriving at the bottom of the page, there was a lack of orientation as to what to do.

They decided to test bringing the donation form all the way up to the top of the page, under the copy, so that it was right in front of the user as they made the decision to give. They also linked the copy to the form through a call-to-action headline.

They made both of these changes at once and launched an experiment to discover the impact on conversion.

Research Question

We believe that bringing the problem into the headline and streamlining the donation experience for child page visitors will achieve an increase in conversion rate.

Design

C: Control
T1: Treatment #1

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 1.2%
T1: Treatment #1 3.5% 199.1% 95.7%

This experiment has a required sample size of 322 in order to be valid. Since the experiment had a total sample size of 683, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 199.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

At first the results of the test looked flat, or trending slightly downward…

Screen Shot 2021-10-14 at 10.19.21 AM.png

However, splitting the traffic into desktop and mobile revealed something fascinating. The treatment produced a 199% valid lift in conversion for desktop devices, which make up 39% of conversions.…which is being offset by a 51% near-valid decrease in conversion for mobile devices, which make up 61% of conversions. 


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #78943

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.