How does a button on a slide-out impact email acquisitions?
WUNC
Experiment Summary
Ended On: 04/22/2024
In an effort to enhance email acquisitions for a content offer, an experiment was conducted for WUNC, a non-profit news radio station. This experiment was tested on the homepage of their website to determine whether prospects are more likely to sign up for an offer directly within a slideout or by being redirected to a landing page. The aim is to gain insights into the most effective strategy for maximizing email acquisitions.
Research Question
We hypothesize that incorporating the form directly into the slideout will streamline the process, reducing friction and the number of steps required, potentially resulting in a higher number of sign-ups. Conversely, a slideout without a form may decrease the perceived commitment, thereby encouraging more sign-ups after users have gained additional information on the landing page
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.62% | ||
| T1: | Treatment #1 | 0.97% | 55.2% | 100.0% |
This experiment has a required sample size of 5,117 in order to be valid. Since the experiment had a total sample size of 89,767, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 55.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learnings from this experiment are:
1. The treatment, which included a link to direct users to a separate landing page, resulted in a 56.5% increase in Emails Acquired for All Traffic compared to an embedded sign-up form.
2. By requiring users to navigate to a separate landing page to provide their information, the treatment potentially decreased the perceived commitment, leading to more sign-ups. This suggests that after users have gained additional information on the landing page, they are more likely to sign up for a content offer
3. The results of the experiment showed a 100.0% statistical level of confidence, indicating that the increase in email acquisitions was not due to chance.
The key learnings from this experiment can guide decision-making in future experiments. It is recommended to continue testing different sign-up processes to further optimize email acquisitions. Additionally, user convenience and reducing perceived commitment can be important factors to consider when designing and implementing sign-up forms on websites.