Save the Children

How Increasing Autonomy with a Micro-Ask In An Emergency Banner Impacts Donor Conversion

Experiment ID: #170434

Save the Children

Experiment Summary

Ended On: 02/27/2024

For this experiment, we wanted to see if adding a simple yes/no question on a banner could increase donor conversion on the page. By asking a prospective donor if they will donate and then giving them a choice of saying yes or no could increase the likelihood of donating because the donor has autonomy.

Research Question

Adding a micro-ask decision in the emergency banner will effectively increase the overall conversion rates. This is because when users are presented with a Yes/No decision, they are more likely to engage, for the good or bad, rather than to bypass it by scrolling.

Design

C: Control

T1: Treatment

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence
C:	Control	0.71%
T1:	Treatment	1.1%	52.4%	98.0%

This experiment has a required sample size of 4,946 in order to be valid. Since the experiment had a total sample size of 14,039, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

0% increase in traffic
× 52.4% increase in conversion rate
× 0% increase in average gift

Key Learnings

This test aimed to assess the impact of integrating a micro-ask into an emergency banner. Key results from the analysis include: +63% statistically significant lift (99%) in desktop click rate. +77% statistically significant lift (95%) in desktop conversion rate. +174% statistically significant lift (99%) in desktop RPV. The +77% lift in desktop conversion rate primarily stems from increased clicks on the banner button. Notably, the variant exhibited a click rate lift of +63%. From a psychological standpoint, individuals are more inclined to engage when they feel a sense of control and agency over their actions. This empowerment instills a sense of investment in the decision-making process, thereby enhancing their likelihood to interact with the banner, manifested in increased clicks. Essentially, by offering users the freedom to choose, we create an experience that resonates with their preferences and motivations. As a result, the observed click rate lift underscores this approach's efficacy in driving user engagement and, ultimately, achieving our desired outcomes. Furthermore, the +174% lift in desktop Revenue per Visitor (RPV) suggests a significant behavioral shift. Users exposed to the micro-ask demonstrated a heightened propensity for larger donations following engagement. While quantifying psychological shifts presents challenges, we hypothesize that engagement with the micro-ask fosters deeper commitment, leading to larger average donations. Partial validation of this hypothesis is observed in the directional lift in donation form conversion rates between treatments (37.5% versus 34.6%). These findings underscore a compelling insight: the introduction of a micro-ask substantially enhances user engagement despite its seemingly inconsequential nature. Moreover, it appears to instill a sense of commitment, resulting in more substantial contributions. This indicates that when users encounter micro-asks, they not only exhibit a higher inclination to engage but also tend to make larger donations, driven by an increased sense of commitment post-engagement.

Experiment Documented by NextAfter

Question about experiment #170434

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.