FamilyLife

How a “complete your gift” popup during a Fiscal Year End Campaign affects donor revenue

Experiment ID: #105252

FamilyLife

FamilyLife® has been committed to helping individuals find biblical help for their marriage and family relationships. Through the Weekend to Remember® marriage getaways, FamilyLife Today® radio broadcasts, The Art of Marriage® video event, and the many other resources and content, God has used FamilyLife to restore hope for millions of couples and transform their lives.

Experiment Summary

Timeframe: 08/24/2022 - 09/04/2022

FamilyLife is a Christian ministry serving families worldwide. For their august Fiscal Year-End campaign, they were seeing a decrease in donor revenue and we wanted to see if we could turn that around in the final days of the campaign. We decided to test a popup what would show if a visitor visited any donation page on their site and chose a gift amount but then abandoned the page before competing their donation. This popup would then fire when they went anywhere else on the site and offer them the prompt to complete their X gift or indicate they weren’t ready to do so. If they chose to, it would reload the donation page so they could complete the transaction.

We split the abandoned donation page traffic 50/50 and ran the experiment for about 11 days.

Research Question

We believe that a “complete your gift” popup targeting visitors who choose a gift amount but abandon the donation page will increase donor conversion.

Design

C: Control
T1: Abandon Donation Popup

Results

 Treatment NameRevenue per VisitorRelative DifferenceConfidenceAverage Gift
C: Control $2.11$111.72
T1: Abandon Donation Popup $2.4917.7% 94.6%$129.19

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was not met so these experiment results are not valid.

Key Learnings

We were only able to record an extremely marginal impact on donor conversion for the popup experience, however the impact on revenue was found in all primary channels to the website. While we were just under a 95% LOC for all traffic, we found a similar increase on all other channels including a 41% increase for direct traffic at a 97% LOC.

The popup reduced friction for these donors, causing them to give a higher amount when they revisited the donation page. Since this ran during the Fiscal Year End campaign, we want to relaunch it in September to measure impact without a high urgency campaign skewing results one way or another.

Other revenue impact measured included:

  • Social visitors had a 77.0% increase in revenue with a 91% level of confidence.
  • Desktop visitors had a 20.7% increase in revenue with an 88% level of confidence.
  • Mobile visitors had a 13.8% increase in revenue with an 88% level of confidence.
  • New visitors had a 16.6% increase in revenue with a 91% level of confidence.
  • Returning visitors had a 17.9% increase in revenue with an 86% level of confidence.


Experiment Documented by NextAfter

Question about experiment #105252

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.