How stating a specific goal in the headline of a pop-up for CYE fundraising affects donor conversion rate
KCBI
Experiment Summary
Timeframe: 12/15/2022 - 12/23/2022
Our CYE fundraising strategy included a pop-up on the Libertas homepage with the goal of delivering the Tuttle Twins series to 200 public schools by raising $50,000. Most visitors to the site want to ensure that we are teaching children the value of liberty so we appealed to this desire with the headline “Help teach Kids the Value of Liberty by Making a Year-End Donation” which is clear and direct. However, we also know that a tangible goal appeals to donors, so we wanted to see if we stated the goal in the headline if it would lift response.
Research Question
We believe that including the goal in the pop-up headline for website visitors will achieve a greater conversion rate of visitors to donors.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Pop-up Control | 0.00% | ||
| T1: | Pop-up Treatment | 4.4% | 100.0% | 98.8% |
This experiment has a required sample size of 91 in order to be valid. Since the experiment had a total sample size of 255, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 100.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment pop-up saw a 4.3% lift in donor conversion rate with a 98.7% level of confidence. This experiment is only based on 6 total gifts, but due to this campaign being very time sensitive and the control pop-up receiving 0 gifts, we decided to roll out the treatment headline to 100% of traffic.
We believe the treatment headline won in this experiment because the goal gave donors something tangible to give to instead of a more broad topic like “teaching kids about liberty”.