How specificity in impact and in the ask affects email conversion rate
Compassion International
Experiment Summary
Ended On: 07/09/2020
We were working with Compassion on a program called “Test and Learn”, where we tested monthly appeals sent to child sponsors. When we reviewed the first email, one key element of persuasion and conversion seemed to be missing: specificity. Two key parts of the email seemed very “broad”, and we hypothesized that specificity could increase conversion.
First, the introduction to the email only broadly acknowledged the relationship between the organization and the sponsor. It thanked them for “deeply caring for children in poverty”. Since child sponsorship at Compassion is 1:1, the recipient had a sponsor (in this case, it was a child named Wayne), and their marketing automation system could use that on a personal basis. We added this detail in to the treatment, and specifically thanked the recipient for being a Compassion sponsor.
Second, the case was made for “poverty” in general using statistics to set up an ask for the “Where Most Needed” fund. One of the difficult things about fundraising to combat poverty is that the need can feel overwhelming. Large statistics can amplify this “overwhelming” factor rather than making it seem like something the donor can impact. So, we took the Where Most Needed fund and positioned it as a way to help meet the additional needs of the sponsor’s specific child, Wayne, revealing that sponsorship doesn’t meet every possible unplanned need.
Our hypothesis was that connecting the donor with a person—particularly one they have a relationship with—rather than a cause or an organization would have a tremendous impact on conversion.
Then, we split the audience to determine which one won…
Research Question
How will adding specificity in impact and in the ask affect email conversion rate?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.73% | ||
| T1: | Treatment #1 | 1.5% | 101.4% | 100.0% |
This experiment has a required sample size of 1,526 in order to be valid. Since the experiment had a total sample size of 20,000, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 101.4% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment, with increased specificity, increased response by 101.4%. This revealed a few powerful insights:
- People might have trouble wrapping their head around a problem expressed generally, or with statistics. This may make the problem seem too broad or difficult to solve. It also may be difficult for the donor to understand how their gift can make any difference at all.
- The Where Most Needed fund, which is a broad, unrestricted fund, seems to be most effectively communicated through the needs of the sponsor’s specific child. Explaining it in abstract might make it difficult to understand. But adding a person—and the extra layer of a person that the reader cares about—might have a multiplier effect.
- Helping a donor understand a specific goal that they can help reach—in this case, 17 respondents—gives them a very tangible thing to accomplish. Otherwise, they might rest on the assumption that someone else will do the job.