FamilyLife

How splitting the sticky slide-out donation ask into two steps impacts donor conversion on FamilyLife’s website

Experiment ID: #195609

FamilyLife

Experiment Summary

Ended On: 02/26/2025

FamilyLife, a nonprofit dedicated to strengthening families through biblically-based resources, uses a persistent sticky donation slide-out on article pages to encourage giving. The Control (current version) features a single message asking users if FamilyLife had provided them with $25 worth of valuable resources and, if so, invites them to donate. Users can select “Yes” to proceed to the donation page or “No” to close the slide-out.

We previously tested this dual-ask approach during FamilyLife’s year-end campaign, where the Treatment led to a 50.9% increase in conversions but did not reach statistical significance, achieving only a 68.9% level of confidence. Despite this, the results were promising enough that FamilyLife chose to implement it for the remainder of the campaign due to the urgency of year-end giving. Given the directional improvement, we wanted to retest this approach in a non-urgent campaign setting to further evaluate its impact.

In this experiment, we tested a modified version (Treatment) that split the question into two steps. The first step only asked if FamilyLife had provided $25 worth of value. If users clicked “Yes,” they were then shown the donation request. If they clicked “No,” the slide-out closed immediately.

We hypothesized that this two-step approach would improve conversion rates by reducing cognitive load, making the initial interaction feel less transactional, and increasing engagement before presenting the donation ask. The experiment was structured as an A/B test with a 50/50 traffic split between the Control (current sticky donate button) and the Treatment (split-question version), measuring whether this adjustment led to higher donor conversions.

Research Question

We believe that splitting the donation ask into two steps will increase conversion rates. By first asking whether FamilyLife has provided value and then following up with a donation request, we reduce cognitive load and create a smoother, less overwhelming user experience, which will lead to higher engagement and more conversions.

Design

C: Control

T1: Treatment #1

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence
C:	Control	0.03%
T1:	Treatment #1	0.06%	65.0%	77.0%

This experiment has a required sample size of 69,116 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

This experiment ran for 30 days. The Treatment (dual ask) led to a 65% increase in donations compared to the Control (single ask). However, the result reached a 77% level of confidence, meaning it did not fully validate at the typical 95% threshold but showed promising potential. The Control (standard sticky slide-out) generated 9 transactions, while the Treatment (dual ask) resulted in 15 transactions. Similar to the previous test conducted during FamilyLife’s high-urgency calendar year-end campaign, we believe the dual ask improved conversions by reducing cognitive overload. By first asking a low-effort question about the value of FamilyLife’s resources, users were more inclined to engage. Those who clicked “Yes” then encountered the donation request in a way that felt more natural and less transactional. The additional step may have also given users more time to process the ask, leading to a higher likelihood of conversion. Given the positive directional impact across multiple tests, we recommend retesting the dual ask in future campaigns to further validate its effect on conversions. Additionally, there are opportunities to explore other optimizations for the sticky donate button, such as presenting a content offer or premium or updating the copy to enhance the value proposition.

Experiment Documented by Ardee Coolidge

Question about experiment #195609

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.