How a higher preselected value on a gift array impacted revenue Experiment ID: #19183

The Fund for American Studies

Experiment Summary

Timeframe: 11/14/2019 - 11/26/2019

In a continued effort to improve average gift in their acquisition efforts, TFAS looked to create a new experience that increased the minimum gift array value from $25 to $50, then chose to apply their “Most Popular” gift array treatment that was tested into that improved donor conversion and revenue previously and move it from the $50 value to the $75 amount.

Research Question

Would increasing the minimum gift and changing to a higher preselected gift array amount increase donor conversion or revenue?

Design

C: Control
T1: $50 minimum

Results

Treatment Name Revenue per Visitor Relative Difference Confidence Average Gift
C: Control $1.95 $41.79
T1: $50 minimum $0.62 -67.9% 97.1% $28.33

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 52.7% decrease in conversion rate
× 32.2% decrease in average gift
67.9% decrease in revenue

Key Learnings

With a 97% level of confidence, we validated a decrease of 67.9% in revenue from this gift array treatment.

This decrease in revenue was really driven by a combination of decreasing donor conversion (which was -55.3%, LoC: 92%) along with a smaller average gift.

An additional interesting observation is that between the two preselected values across the treatments shown the following was true:

  • For the control experience, 50% of the gifts we received in the experiment were at or above the “Most Popular” gift level of $50. Whereas,
  • The treatment experience produced just 16% of the gifts at or above the “Most Popular” gift amount of $75.

With such a small sample, it’s hard to conclusively confirm this, but this leads to suspicion that at least in acquisition, when we ask for a smaller amount up front, that people are inclined to be more generous, whereas if we ask for too much on the first gift, they say “no” altogether.

Conclusion:
For this reason, we have concluded that at least with acquisition, that the combination of starting the gift array at $50 and preselecting $75 as the “Most Popular” gift amount appears to be “too high of a floor” for this audience. Further experimentation should be made with other donation forms, but this combination certainly appears to not be the best approach for acquisition.


Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.