How a high urgency ask on a pop up affects click rate
National Breast Cancer Foundation
The National Breast Cancer Foundation's mission is to provide help and inspire hope to those affected by breast cancer through early detection, education, and support services.
Experiment Summary
Timeframe: 10/26/2017 - 10/31/2017
National Breast Cancer Foundation was in the final days of their BCAM campaign, and wanted to run a test using high urgency copy and a countdown clock on the popup that was running on all pages.
Research Question
Does increasing urgency in copy and with a countdown clock increase click rate to the donation page?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Not high urgency | 6.2% | ||
| T1: | High urgency with countdown clock | 1.8% | -70.8% | 100.0% |
This experiment has a required sample size of 165 in order to be valid. Since the experiment had a total sample size of 73,630, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
70.8% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The high urgency treatment with the countdown clock decreased clicks by 70%. Since the design on the high urgency ad was also different than the control, it’s hard to be sure which specific part of the ad deterred people from clicking, but it’s likely that the new design paired with the very direct ask led only the very most donor-motivated people to click through.