National Breast Cancer Foundation

How a third-party personal recommendation affects email acquisition rate

Experiment ID: #5521

National Breast Cancer Foundation

The National Breast Cancer Foundation's mission is to provide help and inspire hope to those affected by breast cancer through early detection, education, and support services.

Experiment Summary

Timeframe: 10/01/2016 - 10/17/2016

National Breast Cancer Foundation created a free eBook called What Every Woman Needs to Know to give away during Breast Cancer Awareness Month. They created a landing page to advertise this offer, which showed the cover of the book and presented a value proposition to download it.

However, they developed a hypothesis that a personal recommendation from a woman, delivered through an image and a quote, would have more impact on a visitor’s decision to download the eBook than the current image of the eBook.

They designed a treatment featuring an picture of a woman and a personal quote in place of the eBook image, then launched an A/B test to see if it increased landing page conversion.

Research Question

Will a third-party personal recommendation increase email acquisition rate?

Design

C: eBook image
T1: Dora image

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: eBook image 16.4%
T1: Dora image 18.8%14.6% 98.6%

This experiment has a required sample size of 1,934 in order to be valid. Since the experiment had a total sample size of 6,319, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 14.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment, with the picture of Dora and her quote increased email acquisition rate by 14.6%. While this is more of an incremental lift, it had a substantial impact on the number of emails acquired and reduced the cost-per-acquisition.

It also reveals something about the market—people trust people, even if they don’t know them. While the image of the eBook was helpful to clarify the offer in the mind of the visitor, the picture and quote from Dora made them believe that the offer was of greater value, which led to more downloads.


Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #5521

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.