FamilyLife

How a premium offer dear reader CTA on podcast pages impacts donor conversion

Experiment ID: #160295

FamilyLife

FamilyLife® has been committed to helping individuals find biblical help for their marriage and family relationships. Through the Weekend to Remember® marriage getaways, FamilyLife Today® radio broadcasts, The Art of Marriage® video event, and the many other resources and content, God has used FamilyLife to restore hope for millions of couples and transform their lives.

Experiment Summary

Timeframe: 07/17/2023 - 08/14/2023

FamilyLife has a number of popular podcasts with thousands of visitors a month coming to their site to tune in. In reviewing these podcast pages, we hypothesized that we could provide further value for visitors and increase donor conversion by presenting a premium offer in a “dear listener” message on the page.

In our prior testing, we found that an end-of-article dear reader would increase donor conversion on article pages by reducing the friction of making a gift and increasing the appeal of the donation CTA through more space for value-driven copy. We believed that a dear listener message would achieve the same result by providing a way for listeners to engage in a deeper way with what they were hearing and receive a valuable resource in exchange for their gift.

We selected their most popular premium offer, Seeing Beautiful Again and created a dear-listener message on the podcast pages to promote it. We split traffic evenly and ran the experiment for a full month.

Research Question

We believe that a dear reader premium offer on podcast pages will increase donor conversion

Design

C: Control
T1: Treatment #1

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 0.10%
T1: Treatment #1 0.22%112.8% 91.0%

This experiment has a required sample size of 9,026 in order to be valid. Since the experiment had a total sample size of 13,436, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

The dear listener message improved donor conversion significantly. While we weren’t able to validate this increase to a 95% or greater level of confidence, the lack of a negative impact from the treatment makes us confident that featuring a dear-listener premium offer is more likely than not to improve donor conversion. For our next experiment we will test this premium offer against a different one to see which sort of offer best aligns with the visitors to these podcasts. We will also continue to test the placement of the message on the page to see how that impacts donor conversion and user friction.


Experiment Documented by NextAfter

Question about experiment #160295

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.