How presenting a micro ask before the macro, impacts email conversion from a pop-up on the website
Save the Children
Experiment Summary
Timeframe: 05/17/2021 - 05/27/2021
For the past five months, Save the Children has been testing into the email acquisition pop-up on their website. The majority of these experiments have been focused around messaging and value proposition. But all of these experiments have resulted in no positive or negative impact on conversion. As we continue to test into this feature on the website and look for ways to optimize the experience, we hypothesized that we might be able to engage more website users and ultimately increase name conversion by presenting an easier engagement experience on the pop-up before we ask them to give us their email address.
To test this, we presented a “yes”/”no” question that a user would answer before the email acquisition form field would show. The “yes”/”no” question was the same sentiment as the control – “Would you like to stay informed?”. The “Yes” button was highlighted red. When a person would click on the “Yes” button, an email form would slide out and the copy read, “Great! Please provide your email address to sign up.” If a person clicked “No”, the pop-up would simply disappear.
The control presented the same kind of email acquisition opportunity, but in the form of a statement – “Sign up to stay informed”.
Research Question
Will a micro ask before the macro ask increase email acquisition from the pop-up on the website?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.9% | ||
| T1: | Yes/No Micro Ask | 2.6% | 35.4% | 100.0% |
This experiment has a required sample size of 3,643 in order to be valid. Since the experiment had a total sample size of 52,535, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 35.4% increase in conversion rate
× 0% increase in average gift
Key Learnings
After running the experiment for 10 days, we found that the “Yes”/”No” micro ask increased email acquisition by 35%! This valid lift happened on both desktop and mobile devices. In fact, mobile devices had the most significant valid increase in name conversion of 42% (99% LoC). This experiment shows us the importance of engaging people through an easy entrance point results in more people likely to convert. This micro ask increases a persons likelihood and motivation to complete the macro ask.