Avila Foundation

How using a humanized sticky bar during a high-urgency campaign impacts donation conversion rates.

Experiment ID: #120483

Avila Foundation

Experiment Summary

Timeframe: 12/20/2022 - 01/04/2023

During Avila Institute’s year-end campaign, we wanted to capitalize on traffic to one of their most popular websites, SpiritualDirection.com, and run a 3-way experiment to see whether a sticky bar made an impact on donations.

For this experiment, our control was no sticky bar; variant one had a sticky bar with just copy; variant 2 had a sticky bar with a picture of Dan Burke, the founder of Avila. Both variant 1 and 2 had the exact same copy. We ran this sticky bar on all pages on the website.

Research Question

We believe that adding a sticky bar for website visitors will achieve a change in donor conversion rates.

Design

C: Control
T1: Sticky Bar - General Sticky Bar
T2: Sticky Bar - Humanized with Dan's Photo

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.30% $0.00
T1: Sticky Bar - General Sticky Bar 0.36% 20.0% 50.0% $0.00
T2: Sticky Bar - Humanized with Dan's Photo 0.58% 92.2% 99.3% $0.00

This experiment has a required sample size of 6,180 in order to be valid. Since the experiment had a total sample size of 24,875, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 92.2% increase in conversion rate
× 0% increase in average gift

Key Learnings

While we expected a sticky bar to outperform no sticky bar, the results of adding in Dan’s image were really interesting. The Dan image version of the stick bar had a 92% increase in conversion rates compared to both the control and variant 1. We believe that this is due to Dan’s notoriety within the Avila community; everyone knows who he is, what he looks like, and by adding an image we humanized the message a little more, giving donors the perception that it was really Dan asking for urgent help during the campaign.

Notable changes to other metrics due to the experiment:

  • Revenue experienced a 89.4% increase with a 94% level of confidence.

Observations of the impact on other visitor segments in the experiment:

  • (Other) visitors had a 129.3% increase in donations with an 90% level of confidence.
  • Email visitors had a 57.2% increase in donations with an 86% level of confidence.
  • Organic Search visitors had a 423.5% increase in donations with a 90% level of confidence.
  • Male visitors had a 227.5% increase in donations with an 87% level of confidence.
  • 35-44 visitors had a 1.3% increase in donations with a 96% level of confidence.
  • Desktop visitors had a 54.1% increase in donations with an 87% level of confidence.
  • Mobile visitors had a 123.0% increase in donations with a 99% level of confidence.
  • New visitors had a 209.2% increase in donations with a 99% level of confidence.


Experiment Documented by NextAfter

Question about experiment #120483

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.