How clarifying the value proposition through inductive copy affects donor conversion Experiment ID: #6317

Heritage Action for America

Experiment Summary

Timeframe: 2/10/2017 - 3/16/2017

Heritage Action for America was running a petition campaign to confirm Supreme Court nominee Neil Gorsuch. After the petition was signed, the signer was presented with a donation ask. Though they were acquiring thousands of signatures, donor conversion was very low, so they wanted to launch an experiment to increase response. Upon review of the page, they noticed that the headline was relatively self-contained—meaning that it provided confirmation of the signature and did not entice the reader to read any more of the copy.

They created a treatment that still thanked the visitor for signing, but immediately teased them by offering “three reasons” why their signature was critical, to keep them reading down the page. They hypothesized that if they could capture the reader’s attention for just a second longer, they could increase donor conversion by drawing the reader into the body copy.

They launched an A/B test to determine a winner.

Research Question

Will clarifying the value proposition through inductive copy increase donor conversion?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: Original
T1: Inductive Copy

Results

Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Original 0.12% $30.38
T1: Inductive Copy 0.29% 146.9% 97.5% $27.75

This experiment has a required sample size of 5,293 in order to be valid. Since the experiment had a total sample size of 13,792, and the level of confidence is above 95%, the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 146.9% increase in conversion rate
× 8.6% decrease in average gift
125.6% increase in revenue

Key Learnings

The new copy produced a massive 146.9% lift in donor conversion. This illustrates the principle of the “micro-yes”: that the headline is only one part of the conversion process, and must work in tandem with the body copy if we want the reader’s eye to be drawn down the page so that they can absorb the value proposition to donate. If the reader feels a sense of completion after reading the headline, then their potential to donate has been lost. However, if the headline provokes the reader to further engage with the copy, then it increases the conversion horizon and increases the possibility that the reader will become a donor.

With a few weeks until the Supreme Court confirmation hearings, Heritage Action’s team quickly moved to build on this learning with a new experiment.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.