How a Pop-Up on the Journal Page Increased Follows
CaringBridge
Experiment Summary
Timeframe: 03/29/2023 - 04/07/2023
CaringBridge is a platform where users can update friends and family about their health journey. In the journal view of a user’s page, CaringBridge wanted to capture users who were browsing the user’s entries but did not follow them. We hypothesized that a simple popup with value copy about staying updating and informed on the user’s health journey would increase the number of follows.
Research Question
We believe that a popup with value copy for readers on a user’s page, but who do not follow them will achieve an increase in follows because the copy helps impart the importance of staying updating and how they can offer support by following.
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
---|---|---|---|---|---|
C: | Control | 7.4% | $0.00 | ||
T1: | Follow Nudge | 17.7% | 138.8% | 100.0% | $0.00 |
This experiment has a required sample size of 79 in order to be valid. Since the experiment had a total sample size of 47,587, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 138.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that adding a pop-up with value copy about staying updated and informed on the user’s health journey can significantly increase the number of follows on a user’s page. The treatment group saw a 138.8% increase in Follow Site (Journal Pages) for All Traffic with a 100.0% statistical level of confidence. This improvement was seen across all visitor types, including desktop, mobile, tablet, new, returning, direct, email, organic search, and paid search visitors. However, visitors aged 45-54 had a 221% decrease in follow site (journal pages) with a 97% level of confidence.
In future experimentation, it is essential to consider the age group that did not respond positively to the pop-up and adjust the messaging accordingly. Additionally, testing different variations of the pop-up, such as other types of value copy or presenting it at various points during the user’s journey on the platform, could reveal even more effective ways to increase follows. Overall, the results of this experiment demonstrate the value of reminding users, not only that they weren’t already following, but also the important of how their follow supports the user.
Question about experiment #143732
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.