Canadian Taxpayers Federation

How reducing the options on a gift array impacted donations

Experiment ID: #128829

Canadian Taxpayers Federation

A not-for-profit citizen's advocacy group dedicated to lower taxes, less waste and accountable government.

Experiment Summary

Timeframe: 02/16/2023 - 03/09/2023

Because we had seen promising results on a similar test, we wanted to test reducing the number of options on a gift array for an organization’s French-speaking audience to see if they would also have an increase in conversion. The control donation page offered 5 donation amounts and an optional “Other” box to choose a gift of any amount. The treatment presented only a $10 ask and the “Other” field. We ran the test for 3 weeks splitting traffic evenly and measured results on donations and revenue.

Research Question

We believe that reducing the number of gift array options for instant donation page visitors will achieve an increase in donor conversion because there is less friction in the decision-making process.


C: Control
T1: $10


 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 4.5%$0.00
T1: $10 9.7%116.3% 96.7%$0.00

This experiment has a required sample size of 189 in order to be valid. Since the experiment had a total sample size of 426, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 116.3% increase in conversion rate
× 0% increase in average gift

Key Learnings

What we found after running the test for 3 weeks was the treatment produced a 116% lift in donations compared to the control with 96% level of confidence so the results are valid. We also saw a 136% lift in revenue with 98% level of confidence. However, we still had less than the minimum number of required transactions for these results to be statistically significant. We believe these results are due to the decrease in decision making because the donor does not have to spend additional time deciding how much to give. In future testing, we will want to test other gift amounts presented as an isolated option to see if the low $10 amount was the reason for the increase or if it was solely a result of reducing the number of options.

Experiment Documented by Rebekah Josefy
Rebekah Josefy is an Optimization Director at NextAfter.

Question about experiment #128829

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.