How speaking directly to the donor affects conversion Experiment ID: #16355

Texas Public Policy Foundation

Experiment Summary

Timeframe: 4/15/2019 - 6/10/2019

Texas Public Policy Foundation was running a property tax survey for Texans, followed by a donation ask. The campaign was acquiring donors at a decent rate, but with the volume of people taking the survey, they thought they could convert more. When they looked at the donation page, they noticed that the first question was a “grand statement”, followed by a bulleted list of groups that are affected by property taxes. Also, the text asked them to be a part of a “growing movement”—and previous research had shown that when prospective TPPF donors were shown language about being a part of a larger group, it reduced results. The TPPF team thought that the readers might not empathize with every group listed here, and that a more direct, personal approach might convert more donors. They created a treatment that spoke directly to the reader, only addressing the needs that they knew a survey-taker would have: home ownership, personal tax burden, and cost of living.

They launched a test between the two pages to determine a winner.

Research Question

How will speaking directly to the donor affect conversion?

Design

C: Control
T1: Copy Treatment - "You" centered

Results

Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.86% $0.00
T1: Copy Treatment - "You" centered 1.6% 85.3% 96.3% $0.00

This experiment has a required sample size of 1,726 in order to be valid. Since the experiment had a total sample size of 3,919, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 85.3% increase in conversion rate
× 0% increase in average gift

Key Learnings

With similar amounts of traffic, the treatment produced 85.3% more donations. This reinforced the fact that TPPF donors are more concerned with personal impact than the impact of groups, and showed the power of speaking in a “one-to-one” manner.

Once the winner is rolled out, the results on the campaign will drastically improve thanks to this experiment.


Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.