How a different value proposition affects email acquisition on a homepage ad
NextAfter
Experiment Summary
Timeframe: 12/08/2020 - 12/30/2020
In this experiment, we were promoting a new ebook focused on fundraising ideas that can help increase revenue during the last week of the year. We had two main ideas as to how to message the advertisement.
In version A, we used the homepage ad itself to illustrate how fundraisers can use a similar homepage ad to promote their year-end campaign.
In version B, the messaging emphasis focused on the urgency around the biggest giving week of the year being right around the corner.
Research Question
We believe that a value proposition that conveys more urgency and focuses on the revenue at stake for homepage visitors will achieve more ebook downloads.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | "Are YOU Maximizing your homepage traffic?" | 2.5% | ||
| T1: | "Biggest Giving Week" | 3.8% | 50.6% | 84.4% |
This experiment has a required sample size of 1,453 in order to be valid. Since the experiment had a total sample size of 1,496, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
We didn’t quite hit our desired 95% level of confidence, but the result was a strong indicator that it was more relevant to emphasize the urgency around the last week of the year – leading to a 50% increase in conversions. As noted below, it was especially effective for website visitors that already had some affinity towards us and our content.
Observations of the impact on other visitor segments in the experiment:
- Returning visitors had a 77.7% increase in all form submissions with an 89% level of confidence.
- Email visitors had a 213.3% increase in all form submissions with a 96% level of confidence.
- Paid Social visitors had a 66.7% increase in all form submissions with a 99% level of confidence.