Scouts

How adding a header image on an instant donation page affects donor conversion rate

Experiment ID: #92615

Scouts

Experiment Summary

Timeframe: 04/21/2022 - 05/15/2022

Boy Scouts of America is running the National Scouting Survey as a digital acquisition offer through paid Facebook advertising. This survey was crafted to get the feedback and opinions of BSA from former and current Scouts along with other people that have connections with Scouting.

We ran this experiment on the instant donation page to test if having a header image affects donor conversion rate. BSA previously had header images on all donation pages, so we wanted to see how a page without an image would perform.

 

Research Question

We believe that removing the header image on this instant donation page for prospective donors will achieve a higher donor conversion rate.

Design

C: One-column with the image
T1: One-column without the image

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: One-column with the image 0.52% $0.00
T1: One-column without the image 2.4% 367.1% 97.7% $0.00

This experiment has a required sample size of 312 in order to be valid. Since the experiment had a total sample size of 793, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 367.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

The removal of the header image resulted in a 367.1% increase in donations with a 98% level of confidence. This informs us that removing the image removes an element of friction and/or distraction from the page causing users to be more likely to give. We believe by removing the header image that the page seems shorter and less of a time commitment.

Another notable impact from this experiment was for mobile users visiting this page. We saw 201% increase in donor conversion rate with a level of confidence of 84.7%. We believe removing the image provided the mobile user with a better page experience.

All in all, we believe the header image does not provide any value to the donation page, but we would like to further test this on other instant donation pages for more digital acquisition offers.

 


Experiment Documented by NextAfter

Question about experiment #92615

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.