How a sticky donate button with a campaign specific CTA effects donor conversion during a high urgency campaign
Focus on the Family
Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.
Experiment Summary
Timeframe: 06/07/2022 - 06/22/2022
Focus on the Family runs a high urgency campaign every year centered around pro-life issues. This year, they wanted to test if adding a sticky donate button to article pages with a call to action mirroring the ongoing high urgency campaign would result in higher donor conversions than a similar button with a generic call to action. It is worth noting that this experiment was limited to a segmented audience that used user data to identify if a user had prior pro-life interests. For the control, a generic donate button (with a pulsing heart icon) was placed on article pages that matched the data segmentation. For the treatment, a sticky button with the words ” Save a Baby” (with a pulsing baby icon) was placed on article pages that matched the data segmentation.
Research Question
We believe that a CTA on a sticky donate button related to an ongoing high urgency campaign will result in a higher number of donations, than a generic donate button.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Sticky Donate Button | 0.07% | $0.00 | ||
| T1: | Sticky Save a Baby Button | 0.12% | 55.0% | 95.6% | $0.00 |
This experiment has a required sample size of 43,351 in order to be valid. Since the experiment had a total sample size of 91,296, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 55.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
During this experiment that had 91,000 samples over a two week period, we saw a 55% increase for the treatment with a 95.5% Level of Confidence. In this case, this experiment validated for the hypothesis. Key takeaways from this experiment are that matching the CTA of the button to the segmented audience during a high urgency campaign can result in a greater conversion.
Next steps for the experiment would be to test his same hypothesis using the same data segments but do so during a non-high urgency campaign to see if properly segmented users are still converted at a higher frequency with a specific CTA.