How stating a specific goal in the headline of a pop-up for CYE fundraising affects donor conversion rate


Experiment Summary

Timeframe: 12/15/2022 - 12/23/2022

Our CYE fundraising strategy included a pop-up on the Libertas homepage with the goal of delivering the Tuttle Twins series to 200 public schools by raising $50,000. Most visitors to the site want to ensure that we are teaching children the value of liberty so we appealed to this desire with the headline “Help teach Kids the Value of Liberty by Making a Year-End Donation” which is clear and direct. However, we also know that a tangible goal appeals to donors, so we wanted to see if we stated the goal in the headline if it would lift response.

Research Question

We believe that including the goal in the pop-up headline for website visitors will achieve a greater conversion rate of visitors to donors.


C: Pop-up Control
T1: Pop-up Treatment


 Treatment NameConv. RateRelative DifferenceConfidence
C: Pop-up Control 0.00%
T1: Pop-up Treatment 4.4%100.0% 98.8%

This experiment has a required sample size of 91 in order to be valid. Since the experiment had a total sample size of 255, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 100.0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment pop-up saw a 4.3% lift in donor conversion rate with a 98.7% level of confidence. This experiment is only based on 6 total gifts, but due to this campaign being very time sensitive and the control pop-up receiving 0 gifts, we decided to roll out the treatment headline to 100% of traffic.

We believe the treatment headline won in this experiment because the goal gave donors something tangible to give to instead of a more broad topic like “teaching kids about liberty”.

Experiment Documented by NextAfter

Question about experiment #54022

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.