How a change to the donation page headline affects conversion
Texas State Historical Association
Experiment Summary
Timeframe: 10/12/2015 - 10/19/2015
As part of their latest email acquisition campaign, the Texas State Historical Association created an instant donor conversion page after a visitor’s email address had been acquired. This page began by thanking the visitor for giving their email address and then launched into a pitch for the visitor to become a member.
Sometimes the smallest changes to a page can have the biggest impact. The first element on the conversion page is the headline. If this headline does not arrest the visitor’s attention or does not address why they are there, many visitors will not make it past this element on the screen. In an attempt to increase the number of donors acquired, we decided to create a new treatment that would more closely align the headline with what we believed to be the ideal value proposition for the page.
Research Question
Which headline treatment will result in the greatest number of donors and revenue?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | "Join the TSHA" Headline | 2.5% | ||
| T1: | "Get Insider Access" Headline | 0.89% | -65.0% | 96.1% |
This experiment has a required sample size of 463 in order to be valid. Since the experiment had a total sample size of 1,070, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 65.0% decrease in conversion rate
× 20.0% increase in average gift
58.0% decrease in revenue
Key Learnings
The resulting treatment resulted in a 62% decrease in the number of donors acquired. The new headline, “Get insider access to the most exclusive Texas History Club in the Nation,” was framed around what the visitor would get when they join. However, what we added in value, we lost in clarity.
The original headline “Do you love Texas history? Join the Texas State Historical Association today!” more clearly stated the purpose of the page. The initial statement also allowed the visitors the self-select whether the page was right for them based upon their love for Texas history. These factors created an experience that proved to be more successful at converting visitors to members.