How does adding a humanized sticky bar to an instant donation page affect instant donation pages?
American Cornerstone Institute
Experiment Summary
Timeframe: 06/23/2022 - 08/01/2022
In an effort to increase donation rates for an ebook acquisition offer, we wanted to test if adding a humanized sticky bar to the instant donation would connect to prospects and encourage more people to give.
We ran this experiment for 36 days, splitting traffic 50/50 between the control and the treatment.
Research Question
We believe that adding a humanized sticky bar to an instant donation page will increase instant donation rates for prospects who receive an ebook acquisition offer.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 4.7% | $0.00 | ||
| T1: | StickyBar | 4.9% | 3.7% | 19.9% | $0.00 |
This experiment has a required sample size of 115,282 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
After running the experiment for 36 days, we observed a 3.7% lift in instant conversion rates with a 19.8% level of confidence, meaning that there were not enough gifts to validate these results.
It does not appear that in this instance adding a humanized sticky bar inspired significantly more prospective donors to give, however the low number of samples and the resulting low level of confidence in these results prevent us from drawing definitive conclusions one way or the other.
We will look for other opportunities to run this experiment in an effort to validate our hypothesis.