How adding a header image on an instant donation page affects donor conversion rate
Scouts
Experiment Summary
Timeframe: 04/21/2022 - 05/15/2022
Boy Scouts of America is running the National Scouting Survey as a digital acquisition offer through paid Facebook advertising. This survey was crafted to get the feedback and opinions of BSA from former and current Scouts along with other people that have connections with Scouting.
We ran this experiment on the instant donation page to test if having a header image affects donor conversion rate. BSA previously had header images on all donation pages, so we wanted to see how a page without an image would perform.
Research Question
We believe that removing the header image on this instant donation page for prospective donors will achieve a higher donor conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | One-column with the image | 0.52% | |||
| T1: | One-column without the image | 2.4% | 367.1% | 97.7% |
This experiment has a required sample size of 312 in order to be valid. Since the experiment had a total sample size of 793, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 367.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
The removal of the header image resulted in a 367.1% increase in donations with a 98% level of confidence. This informs us that removing the image removes an element of friction and/or distraction from the page causing users to be more likely to give. We believe by removing the header image that the page seems shorter and less of a time commitment.
Another notable impact from this experiment was for mobile users visiting this page. We saw 201% increase in donor conversion rate with a level of confidence of 84.7%. We believe removing the image provided the mobile user with a better page experience.
All in all, we believe the header image does not provide any value to the donation page, but we would like to further test this on other instant donation pages for more digital acquisition offers.