How the ask amount in an email fundraising appeal affects clickthrough rate
Harvest Ministries
Harvest Christian Fellowship exists to bring Christians closer to God and to bring nonbelievers to a saving relationship with Him by showing how God's Word and faith in Him are applicable and relevant to everyday life.
Experiment Summary
Timeframe: 05/15/2016 - 05/24/2016
Harvest Ministries sent their mid-month fundraising appeal and wanted to test the size of the gift ask in the copy. This appeal made the direct ask in the copy, and to mirror the direct mail, suggested a gift size of $35, $45, or more. They hypothesized that asking for a larger amount might increase average gift size.
As they were setting this test up, they came up with a new hypothesis — maybe the specific ask, though it worked in direct mail, was too specific. They developed a new hypothesis that asking for a “generous gift of any amount” would increase conversion and overall revenue.
But first, they would measure clickthrough rate to see which variant, if any, drove more visitors to the landing page.
They set up the A/B/C test to test the control against two treatments.
Research Question
Will a higher ask amount or a less specific ask for a “generous gift” increase clickthrough rate on an email fundraising appeal?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | $35 ask | 0.61% | ||
| T1: | $50 ask | 0.55% | -8.7% | 80.2% |
| T2: | Generous gift | 0.81% | 34.2% | 100.0% |
This experiment has a required sample size of 11,099 in order to be valid. Since the experiment had a total sample size of 203,864, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
34.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The less specific ask generated a 34.2% increase in clickthrough rate. This showed that in terms of motivating a reader to click, less specific is better. However, a click doesn’t necessarily indicate a motivation to give. That’s why Harvest also closely monitored the entire funnel to see how the suggested gift array affected donor conversion and revenue.