How clarifying the value proposition through inductive copy affects donor conversion
Heritage Action for America
Experiment Summary
Timeframe: 02/10/2017 - 03/16/2017
Heritage Action for America was running a petition campaign to confirm Supreme Court nominee Neil Gorsuch. After the petition was signed, the signer was presented with a donation ask. Though they were acquiring thousands of signatures, donor conversion was very low, so they wanted to launch an experiment to increase response. Upon review of the page, they noticed that the headline was relatively self-contained—meaning that it provided confirmation of the signature and did not entice the reader to read any more of the copy.
They created a treatment that still thanked the visitor for signing, but immediately teased them by offering “three reasons” why their signature was critical, to keep them reading down the page. They hypothesized that if they could capture the reader’s attention for just a second longer, they could increase donor conversion by drawing the reader into the body copy.
They launched an A/B test to determine a winner.
Research Question
Will clarifying the value proposition through inductive copy increase donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Original | 0.12% | $30.38 | ||
| T1: | Inductive Copy | 0.29% | 146.9% | 97.5% | $27.75 |
This experiment has a required sample size of 5,293 in order to be valid. Since the experiment had a total sample size of 13,792, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 146.9% increase in conversion rate
× 8.6% decrease in average gift
125.6% increase in revenue
Key Learnings
The new copy produced a massive 146.9% lift in donor conversion. This illustrates the principle of the “micro-yes”: that the headline is only one part of the conversion process, and must work in tandem with the body copy if we want the reader’s eye to be drawn down the page so that they can absorb the value proposition to donate. If the reader feels a sense of completion after reading the headline, then their potential to donate has been lost. However, if the headline provokes the reader to further engage with the copy, then it increases the conversion horizon and increases the possibility that the reader will become a donor.
With a few weeks until the Supreme Court confirmation hearings, Heritage Action’s team quickly moved to build on this learning with a new experiment.