How adding intentional name capture links in a high traffic page affects email conversion rate
The Navigators
Experiment Summary
Timeframe: 02/10/2023 - 03/30/2023
One of The Navigators’ most popular resources is its Bible reading plans. We noticed that this particular resource page had a large amount of organic traffic. The existing page setup prioritized direct PDF downloads, but this lack of forms on the page meant that The Navigators’ wasn’t collecting emails or names of those who downloaded these resources. There was an email capture form at the bottom of the page but it’s likely that once someone downloaded a PDF, they would not scroll to the bottom of the page to fill out a form.
We hypothesized that by keeping the page layout the same, but just swapping out the PDF links in favor of a landing page link to each reading plan—would allow the Navigators to keep distributing this resource while also collecting emails in the process.
Additionally, we only targeted those who were first time visitors to the site so as not to diminish ministry impact for the organization.
Research Question
We believe that linking to a landing page instead of a PDF download for users interested in Bible reading plans will allow The Navigators to take advantage of the opportunity for email acquisition.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 2.7% | $0.00 | ||
| T1: | Treatment | 14.3% | 433.9% | 100.0% |
This experiment has a required sample size of 44 in order to be valid. Since the experiment had a total sample size of 3,574, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 433.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that swapping out PDF downloads for a landing page link is a more effective approach to acquire email addresses from users interested in Bible reading plans. The A/B test demonstrated a 433% increase in email acquisition with a 100% level of confidence.
Moreover, the experiment also revealed that the new approach was successful across various visitor types such as mobile, desktop, organic search, male, female, and varying age ranges. The highest increase in email acquisition was seen among new visitors, organic search visitors, and female visitors.
The key metric that The Navigators were concerned about was whether this funnel process would mean that the number of those who wanted to get the resource would decrease.
When looking at the control, we had 596 get the instant PDF. In the treatment we saw 64% of those who clicked the link, followed through in filling out the form to get the resource. Meaning, most people actually took the next step to give their information in exchange for the resource. Because we targeted first time visitors, these are likely new names acquired for the organization.