How visually presenting the donate button differently impacts traffic to the donation page Experiment ID: #21357

Care Net

Experiment Summary

Timeframe: 5/20/2020 - 6/12/2020

On Care Net’s primary navigation, they provide a link that says “Donate”. This link takes people to their donation page where people can go and make a gift to support the organization. We hypothesized that presenting the link as a button and as an animated button might increase visual presence and ultimately increase traffic to the donation page resulting in more gifts and revenue. To test this, we developed two variations. The first variation was simply making the donate link, look like a button. For the second variation, we added subtle animation to the button that had the heart of the Care Net logo pulse once and the effect ripple across the rest of the button.

Research Question

Can we increase traffic to the donation page by visually changing the look of the donate link in the primary navigation?

Design

C: Control
T1: Regular Button
T2: Pulsing Button

Results

Treatment Name Click Rate Relative Difference Confidence Average Gift
C: Control 1.1% $0.00
T1: Regular Button 0.70% -36.0% 98.5% $0.00
T2: Pulsing Button 0.97% -12.1% 55.5% $0.00

This experiment has a required sample size of 6,735 in order to be valid. Since the experiment had a total sample size of 20,362, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    36.0% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

After running the experiment for two weeks, we found that by changing the donate link to a button negatively impacted clicks to the donation page. Additionally, the pulsing donate button had neither a positive or negative impact on clicks. We weren’t able to validate the experiment off of donor conversion on the donation page, but we did we a directional 17% decrease in conversion from variation one – which is ultimately due to the significant decrease in traffic going to the page.

We hypothesize that the donate button saw such a negative impact in clicks because the use of buttons on their website, and across the majority of websites, is overused. People may actually be turned off by this. We believe this hypothesis to be true because of other testing we have done with the Care Net audience where we would test out of button and instead use hyperlink text. This would reinforce why the control with the underlined word “Donate” performed best in this experiement.


Experiment Documented by...

Courtney Gaines

Courtney is a Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.