Senator John Cornyn

How copy on a donation page affects the force of the value proposition

Experiment ID: #111

Senator John Cornyn

Experiment Summary

Timeframe: 04/20/2014 - 04/30/2014

John Cornyn has served as a U.S. Senator from Texas for more than 14 years and is the current Senate Minority Whip for the 113th Congress. As election season begins, a torrent of traffic comes to JohnCornyn.com. This traffic includes likely voters looking to confirm their affiliation, donors looking to support the Senator, and people seeking information about all of the candidates. The Senator Cornyn’s team wanted to make sure that friction and anxiety were minimized on their main donation page and that credibility and clarity were increased so that the most motivated potential donors would make a gift to support the Cornyn campaign.

Research Question

By providing a clear and credible reason to give, we can increase both the average gift and conversion rate of the main donation page.

Design

C: No Value Prop Language
T1: Value Proposition on Page

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: No Value Prop Language 0.11%$13.00
T1: Value Proposition on Page 0.38%258.1% 92.5%$328.38

This experiment has a required sample size of 2,556 in order to be valid. Since the experiment had a total sample size of 3,984, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

The new treatment delivered a 258.1% increase in donor conversion, with a statistically valid sample size. Team Cornyn learned a few things about their donor base through this test:

1. Experience matters. Even though the donor might be motivated to get to the donation page, they need to be reminded throughout the process that Senator Cornyn aligns with their core beliefs and is worthy of their donation. Donors to Senator Cornyn are thoughtful — they can’t be expected to just react to a donation opportunity, they must be persuaded along the way.

2. Design matters. Simply adding the copy to the page hurt results. Keeping continuity in the user experience throughout the entire donation process is crucial to a positive lift.


Experiment Documented by Tim Kachuriak
Tim Kachuriak is Chief Innovation and Optimization Officer of NextAfter.

Question about experiment #111

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.