How replacing 3rd party credibility via social validation affects revenue
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 12/16/2014 - 12/26/2014
As part of most campaign landing pages, Heritage has included 3rd party credibility indicators (such as external quotes and media appearances) as a way to reduce anxiety. However, we thought it might be possible to make it more relevant if the 3rd party credibility came from actual donors instead of well known sources. As such, we developed an embeddable widget that would scroll through recent donors.
Research Question
Showing recent donors will add to the credibility of the previously stated value proposition and will foster a herd mentality within the donor’s mind. This will increase conversion rate compared to other 3rd party credibility indicators
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | 3rd Party Credibility | $3.56 | $108.51 | ||
| T1: | Social Widget | $2.27 | -36.1% | 97.9% | $66.12 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 4.8% increase in conversion rate
× 39.1% decrease in average gift
36.1% decrease in revenue
Key Learnings
Seeing recent donors did not carry as much weight as quotes and appearances in well known sources. In addition to using the 3rd party credibility indicators near the donation form, we will also include these facts in the copy and throughout the page design where appropriate.