How creating a “shortcut” to the donation form affects conversion Experiment ID: #2107

Heritage Action for America

Experiment Summary

Timeframe: 9/9/2015 - 9/17/2015

Leading up to their Republican presidential debates, Heritage Action for America ran a money bomb campaign to create more awareness around the event. This campaign was a two week long series with multiple emails all driving to the same landing page.

Previous experiments had proven that longer copy is better at communicating the value proposition to the donor.  However, we had a hypothesis that we may be able to create a shortcut for some donors that did not need as much convincing. The idea was to put a “Donate Now” button in the top right of the page that would jump the potential donors to the donation form at the bottom thus reducing friction.

Research Question

How will removing the “Donate Now” button on a money bomb impact the number of donations and revenue?

Design

C: No Donate Now Button
T1: Donate Now Button

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: No Donate Now Button 3.2%
T1: Donate Now Button 2.3% -28.2% 95.7%

This experiment has a required sample size of 2,632 in order to be valid. Since the experiment had a total sample size of 5,470, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 28.2% decrease in conversion rate
× 34.1% decrease in average gift
52.6% decrease in revenue

Key Learnings

By adding a “Donate Now” button to the top right corner of the page, we ended up reducing the number of donations by 28%. Our hypothesis is that this button prematurely tipped our hand on the purpose of the campaign and made the ask before the visitors were ready to see it. While the long page copy may add friction to the giving process, the added benefit that comes with a strong value proposition is enough to overcome that friction.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.





Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.


Pick your format of choice:

Related Experiments

This experiment is part of a series of experiments aimed at improving overall results. Take a look at some of the other iterations:


Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.