How a third-party personal recommendation affects email acquisition rate
National Breast Cancer Foundation
The National Breast Cancer Foundation's mission is to provide help and inspire hope to those affected by breast cancer through early detection, education, and support services.
Experiment Summary
Timeframe: 10/01/2016 - 10/17/2016
National Breast Cancer Foundation created a free eBook called What Every Woman Needs to Know to give away during Breast Cancer Awareness Month. They created a landing page to advertise this offer, which showed the cover of the book and presented a value proposition to download it.
However, they developed a hypothesis that a personal recommendation from a woman, delivered through an image and a quote, would have more impact on a visitor’s decision to download the eBook than the current image of the eBook.
They designed a treatment featuring an picture of a woman and a personal quote in place of the eBook image, then launched an A/B test to see if it increased landing page conversion.
Research Question
Will a third-party personal recommendation increase email acquisition rate?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | eBook image | 16.4% | ||
T1: | Dora image | 18.8% | 14.6% | 98.6% |
This experiment has a required sample size of 1,934 in order to be valid. Since the experiment had a total sample size of 6,319, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 14.6% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment, with the picture of Dora and her quote increased email acquisition rate by 14.6%. While this is more of an incremental lift, it had a substantial impact on the number of emails acquired and reduced the cost-per-acquisition.
It also reveals something about the market—people trust people, even if they don’t know them. While the image of the eBook was helpful to clarify the offer in the mind of the visitor, the picture and quote from Dora made them believe that the offer was of greater value, which led to more downloads.
Question about experiment #5521
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.