Bill of Rights Institute

How value proposition in a handwritten note about the impact of a higher gift affects revenue

Experiment ID: #54453

Bill of Rights Institute

Experiment Summary

Timeframe: 03/11/2021 - 05/09/2021

The Bill of Rights needed to increase its average gift and in turn its revenue from acquisition, that in turn will decrease the net cost per donor. We were looking for ways to increase the average gift without damaging the number of donations. To encourage higher gifts, we devised concise copy that would show the impact and worth of their $50 gift. Then we used a handwritten note to set it apart and catch the reader’s eye. This was run on three donation pages to acquire the needed samples. All of them were acquisition offers for eBooks that were advertised on social. We had to run it for two month, longer than typically advised, to acquire the needed samples.

Research Question

Showing the direct impact of a higher gift will increase the average gift and revenue.

Design

C: Control
T1: Handwritten

Results

 Treatment NameRevenue per VisitorRelative DifferenceConfidenceAverage Gift
C: Control $1.25$23.96
T1: Handwritten $0.58-53.3% 85.8%$17.75

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.

Key Learnings

This tactic worked primarily on desktop and for new visitors. It increased the revenue of new visitors by 55%, with an $11 increase in average gift. However, it decreased revenue for mobile visitors by 35%. Our hypothesis is that the required placement for mobile led to a less desirable user experience. It also slightly decreased revenue for returning visitors, but not at the same rate as it increase gifts for new visitors. The average gift was only $4 lower for returning visitors. It also decreased the number of gifts.

Our hypothesis is that returning visitors who see this more than once are less likely to give, because this message is no longer new, but is instead a turn off. But new visitors, who haven’t seen this element before, are intrigued and inspired.

We will roll this treatment out to just new visitors and desktop experience. And we will continue to search for other ways to increase giving for returning and mobile, that also works for new visitors and desktop.


Experiment Documented by NextAfter

Question about experiment #54453

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.