How a direct call to action affects clickthrough rate
Harvest Ministries
Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.
Experiment Summary
Timeframe: 03/09/2016 - 03/21/2016
Harvest Ministries was testing the button text on the premium block on their homepage. This block was used to ask for a donation in exchange for a premium offer, in this case, a DVD.
Since the offer was centered around a premium, the button text had been “get your copy”. However, the copy was clear that the DVD was given in exchange for financial support, so the Harvest team decided to test “donate” on the button text. If the motivation of their site visitors was aligned with charitable giving, they hypothesized that more people would click through, as opposed to the more eCommerce-focused call-to-action of “get your copy”.
Research Question
Will a more direct call-to-action increase clickthrough rate?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.50% | ||
| T1: | Treatment 1 | 0.82% | 63.6% | 100.0% |
This experiment has a required sample size of 5,418 in order to be valid. Since the experiment had a total sample size of 117,649, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
63.6% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
This reveals an interesting and powerful learning about Harvest site visitors: they come to the site with some sort of motivation to give. If Harvest site visitors had a lower motivation to give, they would most likely respond to some other copy like “learn more”, since they wouldn’t be prepared to give. However, the response to the “donate” call-to-action reveals that they are not repelled by a direct ask — in fact, they are more likely to take action.