How going direct to a petition impacted email registration rates Experiment ID: #15473

Americans for Prosperity

Experiment Summary

Timeframe: 4/24/2019 - 5/22/2019

Americans for Prosperity launched a quiz that led to a petition, which led to an instant donation page. We found that although the quiz was acquiring emails, that people were dropping off at the point when we asked them to sign the petition, which meant that visitors to the instant donation page were really low. In an attempt to boost both email registration and instant donation pageviews, we wanted to experiment removing the quiz from the process and directly promoting the petition as the first step.

Research Question

Will a simplified funnel, which removes the first step (a quiz) from the funnel improve email registration rates?

Design

C: Economic Quiz -- Freedom
T1: Economic Petition -- Freedom

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Economic Quiz -- Freedom 0.09%
T1: Economic Petition -- Freedom 0.20% 127.7% 100.0%

This experiment has a required sample size of 8,486 in order to be valid. Since the experiment had a total sample size of 250,270, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 127.7% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 99.9% level of confidence, we were able to lift email acquisition rates by cutting the quiz from the process and directly promoting the petition by +127.7%

Subsequently, we did see directional increases in both donor conversions (+151%, LoC: 54%) and revenue (+151%, Loc: 77%), but neither statistic validated through the experiment, since so few gifts ultimately were secured through the offer.

Although the original thought was that the quiz would build momentum towards petition signatures, which could crescendo into generating instant donors and dollars, in practice we found that a more simplified two-step approach was confirmed to produce more emails, and directionally more donors and dollars.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Greg Colunga

Greg is Executive Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.