The impact of using a donor’s name on a post-survey donation page
Care Net
Experiment Summary
Timeframe: 06/22/2020 - 07/20/2020
As we continued to test the impact of using smart content on donation pages with DepositFix, we decided to take our high performing “Abortion in America Survey” Facebook campaign and test smart content on the donation page. Mainly, if someone was visiting the page already in our database, they would see their name on the page. Since they just gave their name on the survey, we thought that the continuity might increase donor conversion.
Research Question
Will seeing your name on a donation page after giving your name in a survey increase your liklihood to donate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 2.5% | ||
| T1: | Deposit Fix Page | 3.3% | 35.1% | 89.8% |
This experiment has a required sample size of 2,871 in order to be valid. Since the experiment had a total sample size of 4,100, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
After running the experiment for one month, we achieved a 35% increase in donor conversion. However, our revenue decreased by 11%. This was due to a smaller average gift on the variation (smart content) page. The control page received two donations that were $250 and multiple that were $100.
With the lower revenue, we may not move forward with smart content on these campaigns. Revenue and donor conversion are both important, but ultimately revenue is the prevailing concern.