How speaking directly to the donor affects conversion
Texas Public Policy Foundation
Experiment Summary
Timeframe: 04/15/2019 - 06/10/2019
Texas Public Policy Foundation was running a property tax survey for Texans, followed by a donation ask. The campaign was acquiring donors at a decent rate, but with the volume of people taking the survey, they thought they could convert more. When they looked at the donation page, they noticed that the first question was a “grand statement”, followed by a bulleted list of groups that are affected by property taxes. Also, the text asked them to be a part of a “growing movement”—and previous research had shown that when prospective TPPF donors were shown language about being a part of a larger group, it reduced results. The TPPF team thought that the readers might not empathize with every group listed here, and that a more direct, personal approach might convert more donors. They created a treatment that spoke directly to the reader, only addressing the needs that they knew a survey-taker would have: home ownership, personal tax burden, and cost of living.
They launched a test between the two pages to determine a winner.
Research Question
How will speaking directly to the donor affect conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.86% | $0.00 | ||
| T1: | Copy Treatment - "You" centered | 1.6% | 85.3% | 96.3% | $0.00 |
This experiment has a required sample size of 1,726 in order to be valid. Since the experiment had a total sample size of 3,919, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 85.3% increase in conversion rate
× 0% increase in average gift
Key Learnings
With similar amounts of traffic, the treatment produced 85.3% more donations. This reinforced the fact that TPPF donors are more concerned with personal impact than the impact of groups, and showed the power of speaking in a “one-to-one” manner.
Once the winner is rolled out, the results on the campaign will drastically improve thanks to this experiment.