NextAfter

Will a specific resource in a dear reader drive more engagement than a general offer?

Experiment ID: #200232

NextAfter

Experiment Summary

Ended On: 10/16/2025

To capture more email subscribers, the team added a “Dear Reader” prompt at the bottom of review pages offering a free resource. The goal was to test whether visitors would be more motivated by a resource directly tied to Plugged In or a broader tool.

Two offers were tested:

Control: A general video series, Praying the Scriptures Over Your Children.
Treatment: A Plugged In–branded ebook guide featuring curated, family-friendly movie picks.

The test explored whether contextual relevance — aligning the offer with the content just consumed — would increase conversions.

We split the website traffic 50/50 between the control and treatment, and ran this test for 30 days.

Research Question

We believe that offering a Plugged In-branded resource in a Dear Reader for visitors to Plugged In’s review webpages will achieve an increase in email sign-ups because people are more likely to engage with a resource when it is closely aligned to the content they’re already consuming. because people are more likely to engage with a resource when it is closely aligned to the content they’re already consuming.

Design

C: Control

T1: Treatment

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence
C:	Control	0.04%
T1:	Treatment	0.03%	-41.4%	97.9%

This experiment has a required sample size of 82,152 in order to be valid. Since the experiment had a total sample size of 227,657, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

0% increase in traffic
× 41.4% decrease in conversion rate
× 0% increase in average gift

Key Learnings

While both offers generated equal click-through rates from the “Dear Reader” prompt, the general series led to significantly more email sign-ups than the Plugged In–branded movie guide.

The control offer produced 49 sign-ups, compared to 29 sign-ups for the Plugged In ebook — that's a 41.4% decrease for the treatment, with a statistically valid confidence level of 97.8%
From what we can see in the data, the click rates on the Dear Reader itself were identical between the Control and the Treatment, indicating equal interest in the offers initially. The drop-off occurred at the conversion stage, suggesting differences in how compelling or valuable the offers felt to users.
Interestingly, users who signed up for the Plugged In ebook were more likely to go on to donate, even though total sign-ups were lower.
7 donations followed the Control offer; 6 followed the Treatment — not a statistically significant difference.
Average gift was higher for the Treatment ($52 vs $40), after removing a $5,000 outlier from the Treatment.
No significant difference in recurring vs one-time gifts.

The key learning is that the Plugged In offer may have drawn a more motivated or mission-aligned subset of users, even if fewer people converted overall. Meanwhile, the parenting-focused offer had broader appeal and stronger top-of-funnel performance, likely due to its deeper emotional resonance around prayer and family. While contextual relevance (matching the brand and content) matters, emotional depth and perceived value may carry more weight in driving conversions. For next steps, we'll want to evaluate the value proposition of the Dear Reader vs the offer pages that each variant pointed to so we can better understand the differences between them and improve future offers.

Experiment Documented by NextAfter

Question about experiment #200232

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.