The Heritage Foundation

How a tabbed donation form affects donor conversion rates on an acquisition donation page

Experiment ID: #168091

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 01/17/2024 - 02/01/2024

As a follow up to previous experiments, we wanted to see if using a tabbed donation form for visitors to an acquisition donation page would improve donor conversion rate and recurring donor acquisition. In previous experiments, we found no difference in one-time giving, but had seen 99% lift in recurring giving. However this was tested on the main donation page where motivation to give is very different than giving through an acquisition funnel.

Research Question

We believe that a tabbed donation form for acquisition funnel donation page visitors will achieve an increase in overall donor conversion rates.

Design

C: Control
T1: Tabbed Donation Form

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 3.5%
T1: Tabbed Donation Form 4.6%29.5% 90.3%

This experiment has a required sample size of 2,755 in order to be valid. Since the experiment had a total sample size of 3,979, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

The key learnings from this experiment are as follows:

1. The treatment, which used a tabbed donation form, resulted in a 29.5% increase in transactions compared to the control group.

2. While the statistical level of confidence (90.3%) is below the generally accepted threshold of 95%, the results are approaching validity. This means that there is a strong indication that the treatment is effective, but further testing is needed to confirm its impact.

3. The previous experiment on the main donation page showed a 99% lift in recurring giving with a tabbed donation form. However, the current experiment on the acquisition donation page did not see a significant difference in recurring giving. This suggests that the motivation to give may differ between the two situations.

Based on these learnings, it is recommended to continue testing the tabbed donation form on acquisition donation pages to further validate its effectiveness. Consider running additional experiments with a larger sample size and longer duration to improve statistical confidence.

Additionally, it may be valuable to explore the impact of different language and messaging on the donation form. Testing variations in messaging can help determine the most effective language that resonates with potential donors and increases conversion rates.

Overall, the key learnings from this experiment suggest that a tabbed donation form has the potential to improve donor conversion rates on acquisition donation pages.


Experiment Documented by NextAfter

Question about experiment #168091

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.