How adding a “micro-yes” decision impacted conversion
The Heritage Foundation
Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.
Experiment Summary
Timeframe: 11/06/2017 - 11/22/2017
The Daily Signal had been running a series of experiments known as the “Dear Reader” for the past 10 months. This experiment added a direct ask to every Daily Signal article and had resulted in significant revenue for the organization. Over the course of the 10 months, we had tested everything from value proposition to placement.
One learning we had recently observed was a boost in our advertising click through rate from Facebook when we incorporated exceptionally long copy. When doing that, Facebook automatically added a “Read More…” link in the ads. One of our hypotheses for this boost in performance was that having this link created an additional micro-yes decision that required people to mentally commit that they wanted to read more; this caused additional buy-in prior to the call to action being made.
We decided to test this hypothesis out on the Daily Signal Dear Reader treatment. We would hide the content related to the direct ask behind a “Read More” button. Our belief is this additional micro-yes would create more buy-in prior to the donation ask.
Research Question
Will creating an additional micro-yes prior to a direct ask increase overall donations?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | All Content Visible | 0.01% | ||
| T1: | Read More Button | 0.01% | -54.2% | 98.5% |
This experiment has a required sample size of 140,190 in order to be valid. Since the experiment had a total sample size of 479,181, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 54.2% decrease in conversion rate
× 0% increase in average gift
Key Learnings
The Read More button resulted in a 54% decrease in donations. This drop in donations was primarily the result of a drop in clicks on the donation button in the ad; the number of people visiting the donation page dropped by 64%. In this situation, the additional decision created more friction than mental buy-in which negatively impacted the results.