How adding an email acquisition form directly to a relevant blog page impacted offer sign ups
Reasons to Believe
Experiment Summary
Timeframe: 07/27/2022 - 08/04/2022
Reasons to Believe built an offer in which users could welcome the organization’s newly appointed CEO by submitting an “encouraging note” to him. The offer was set up in its traditional way: After clicking on an ad or email link, users would go to a landing page where they would read about this opportunity to submit a personal note. They could then write the note and submit it after providing their first name, last name, and email address.
RTB also posted a blog article on the news of naming the successor to their longstanding president and CEO. We wanted to test how adding the same acquisition form below the blog content would impact email submissions. So instead of reading about the opportunity to submit a personal note, users would read about the news through the blog article and be given the opportunity to submit the personal note afterwards.
For the treatment, we added a bolded call-to-action after the blog article and then added the same acquisition form as we used in the control. We wanted to test which presentation of the offer would lead to more email conversions.
Research Question
We believe that adding an offer sign up opportunity directly below a related and important blog article for page visitors will achieve an increase in email conversions.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 25.0% | $0.00 | ||
| T1: | Treatment | 50.0% | 100.0% | 81.0% | $0.00 |
This experiment has a required sample size of 29 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
The treatment saw a 60% lift in email conversions but only with a 72.5% level of confidence. The experiment did not validate. This offer could only run for a limited amount of time since it was news related. But it might be worth testing again in a similar capacity to see if there is in fact a trend in email conversion for a blog-level acquisition opportunity.
The experiment also revealed that returning visitors were twice as likely to convert than new visitors, which tells us that returning visitors are more familiar with RTB blogs and would perhaps get more out of reading a blog article about a new president taking over for the former president. If a similar offer is run again, it could be tested further for returning visitors.