Will adding the nominee’s name to the petition increase conversion rates?
Americans for Prosperity
Experiment Summary
Timeframe: 09/10/2018 - 09/17/2018
Initially, the Open Letter petition was launched with just the title of the nominee (not using Judge Brett Kavanaugh’s name) within the petition itself. We wondered what would happen if we added his name to the petition — would more people be willing to sign the open letter petition?
Research Question
Will adding clarity to the nominee (instead of just simply using the title for the nominee) increase the petition signature conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 66.3% | ||
| T1: | Specific to Kavanaugh | 71.5% | 8.0% | 100.0% |
This experiment has a required sample size of 589 in order to be valid. Since the experiment had a total sample size of 32,993, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 8.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The 8% statistically significant lift was a significant win in an already high performing campaign. It emphasizes the need to focus on adding the most clarity and specificity to the petition signing process. When appropriate, add the names of people, organizations, or causes that the petition is in favor of to increase petition signing conversion rates.