How adding a resource call-to-action button impacts donor conversion on the broadcast page Experiment ID: #21019

Focus on the Family

Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.

Experiment Summary

Timeframe: 4/20/2020 - 5/10/2020

Focus on the Family gets significant traffic to their general broadcast page. This page shows people all of the available broadcasts they can hear/watch. Through previous testing, we learned that we could increase the number of people getting to the specific episode pages by adding a clear CTA to “watch this episode” on the general broadcast page. Knowing that every episode has a resource connected to it, we wanted to test the CTA idea we used increase the number of people watching the episodes to the resources as well. We developed a treatment that presented a CTA button on every episode featured on the general broadcast page that said, “Get today’s featured resource”. That button clicked through to the specific episode page and anchored to the resource.

Research Question

Can we increase donor conversion by presenting the featured resource on the general broadcast page?

Design

C: With Episode CTA
T1: With Episode + Resource CTA

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: With Episode CTA 0.64%
T1: With Episode + Resource CTA 1.1% 66.8% 96.2%

This experiment has a required sample size of 3,568 in order to be valid. Since the experiment had a total sample size of 8,058, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 66.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

Donor conversion increased by 67% on mobile devices when we presented the resourced CTA on the general broadcast page. This tells us that a large volume of people coming to the broadcast page are either specifically looking for those resources and we were able to make the user experience better for them and get them what they were looking for more effectively, or we were able to convert the less motivated traffic into donors by showing them there is a specific resource tied to an episode they may be interested in. We are currently testing this same experience on desktop devices too.


Experiment Documented by...

Courtney Gaines

Courtney is a Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.