How a homepage takeover affects donor conversion rate at calendar year end
International Mission Board
Experiment Summary
Timeframe: 12/30/2022 - 01/04/2023
IMB was running their calendar year end campaign around their Lottie Moon Christmas Offering. To reduce friction at a time of the year when conversion rate is highest, they wondered if turning their homepage into the donation page would increase donor conversion by reducing the necessary steps between the visit and a gift.
They set up a treatment where some donors would see a donation page with a link to “go to IMB.org” at the top of the page instead of the homepage.
To mitigate risk, they only included 25% of the traffic in this experiment.
Research Question
We believe that taking over the homepage with a donation form for main domain visitors` will achieve an increase in conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 19.9% | |||
| T1: | Homepage takeover | 26.1% | 30.6% | 95.3% |
This experiment has a required sample size of 348 in order to be valid. Since the experiment had a total sample size of 1,028, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 30.6% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment, which showed a donation page on the first step, produced a 30.6% increase in conversion rate. This is significant, as experiments like this are often hard to validate at the end of the year, when motivation to give is highest. However, the reduction in friction showed that donors are still falling out of the funnel, even when coming in with high motivation.
This technique should not be overused, but could also be used with specific campaigns.