Americans for Prosperity

How creating a sense of urgency in your ad copy impacted donor conversion rate

Experiment ID: #37909

Americans for Prosperity

Experiment Summary

Ended On: 10/27/2020

As a part of the SCOTUS 2020 confirmation hearings for (now) Justice Amy Coney Barrett, Americans for Prosperity had designed a digital activism opportunity for citizens to sign a letter and send it to their senators to tell them that they approved this nomination and were urging their elected officials to represent them by voting to confirm the nominee when it came time to officially vote.

The letter-signing page, once completed, would redirect signers to a donation page where they would be asked to make a financial gift to support the continued efforts to confirm the nominee.

In reading the control ads, they lacked a sense of urgency. For this nominee, there was a general feeling that it was a foregone conclusion. Telling people that there was still risk if they didn’t take action now was true, but would it have an impact on results?

Research Question

Will creating a sense of urgency and improving clarity increase the ad clickthrough rate, email signup rate, and/or donor conversion rate?

Design

C: Control

T1: Treatment #1

Results

	Treatment Name	Conv. Rate	Relative Difference	Confidence
C:	Control	0.00%
T1:	Treatment #1	0.00%	89.6%	100.0%

This experiment has a required sample size of 476,430 in order to be valid. Since the experiment had a total sample size of 7,271,764, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

0% increase in traffic
× 89.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 100% level of confidence, we observed that this treatment increased donor conversion rate by +89.6%.

Subsequently, we observed every step of the funnel with increased results:

Ad Clickthrough Rate increased +30.9% (LoC: 100%)
Email Signup Rate increased +14.3% (LoC: 100%)

This meant that so many more people made it through the funnel and onto the donation page, which increased results thereafter (in terms of donor quantities). But the conversion rate was increased in our opinion for two reasons:

As noted, the increased volume of people coming through the funnel, and
The momentum was started at the ad level and carried all of the way through to a cohesive donation page that still cast doubt on the confirmation unless they did more right now.

In terms of breaking down the ads themselves, here are some other observations that led to the massive increases in ad clickthrough rate:

We used trigger-based ads—specifically, by getting a photo of President Trump along with the nominee. This caused more people to pause in their feed, which got more people to read the ad copy (which generated motivation).
Using CLARITY to tell people exactly what they can expect post-click, how their information would be kept safe, and that it would take only a minute all worked to mitigate anxiety for the reader to want to take the action today.
Ensuring that we communicate that there is still doubt unless they take action right now was a motivating factor, as well

Experiment Documented by Greg Colunga

Greg Colunga is Chief Operating Office at NextAfter.

Question about experiment #37909

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.