Focus on the Family

How visually changing the “Give Monthly” button impacts traffic to the donation page

Experiment ID: #21354

Focus on the Family

Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.

Experiment Summary

Timeframe: 5/26/2020 - 6/12/2020

On the Focus on the Family website, they present two options for people to give, “Donate” and “Give Monthly”. We hypothesized that we could increase traffic to the monthly donation page by visually setting off the “Give Monthly” button. To do this, we created a subtle animation on the “Give Monthly” button that used a heart icon that pulsed once and the effect would then ripple across the rest of the button. We split the traffic 50/50 and ran this experiment across the website’s entire navigation.

Research Question

Would subtle animation on the Give Monthly donate button increase traffic to the monthly giving page?

Results

Treatment Name Click Rate Relative Difference Confidence
C: 0.25%
T1: Pulsing Monthly Button 0.31% 20.3% 99.3%

This experiment has a required sample size of 80,708 in order to be valid. Since the experiment had a total sample size of 307,742, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    20.3% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The subtle animation on the “Give Monthly” donate button increase traffic to the monthly giving page by 19%! We weren’t able to validate this experiment off of monthly donations because the conversion rate on the page is too low to validate.

The subtle animation on the treatment caught people’s eye causing them to click through to the donation page. We hypothesize that this tactic may be a novelty. We will continue to monitor it’s effectiveness and retest the approach again in two months to see if the novelty has worn off, or if the subtle animation really is a longer term tactic to increase clicks to the donation page.


Experiment Documented by Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.

Question about experiment #6551

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.