Save the Children

How presenting a micro ask before the macro, impacts email conversion from a pop-up on the website

Experiment ID: #62417

Save the Children

Experiment Summary

Timeframe: 05/17/2021 - 05/27/2021

For the past five months, Save the Children has been testing into the email acquisition pop-up on their website. The majority of these experiments have been focused around messaging and value proposition. But all of these experiments have resulted in no positive or negative impact on conversion. As we continue to test into this feature on the website and look for ways to optimize the experience, we hypothesized that we might be able to engage more website users and ultimately increase name conversion by presenting an easier engagement experience on the pop-up before we ask them to give us their email address.

To test this, we presented a “yes”/”no” question that a user would answer before the email acquisition form field would show. The “yes”/”no” question was the same sentiment as the control – “Would you like to stay informed?”. The “Yes” button was highlighted red. When a person would click on the “Yes” button, an email form would slide out and the copy read, “Great! Please provide your email address to sign up.” If a person clicked “No”, the pop-up would simply disappear.

The control presented the same kind of email acquisition opportunity, but in the form of a statement – “Sign up to stay informed”.

Research Question

Will a micro ask before the macro ask increase email acquisition from the pop-up on the website?


C: Control
T1: Yes/No Micro Ask


 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 1.9%
T1: Yes/No Micro Ask 2.6%35.4% 100.0%

This experiment has a required sample size of 3,643 in order to be valid. Since the experiment had a total sample size of 52,535, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 35.4% increase in conversion rate
× 0% increase in average gift

Key Learnings

After running the experiment for 10 days, we found that the “Yes”/”No” micro ask increased email acquisition by 35%! This valid lift happened on both desktop and mobile devices. In fact, mobile devices had the most significant valid increase in name conversion of 42% (99% LoC). This experiment shows us the importance of engaging people through an easy entrance point results in more people likely to convert. This micro ask increases a persons likelihood and motivation to complete the macro ask.

Experiment Documented by Courtney Gaines
Courtney Gaines is Vice President at NextAfter.

Question about experiment #62417

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.