How specificity in impact and in the ask affects email conversion rate Experiment ID: #21672

Compassion International

Experiment Summary

Ended On: 7/9/2020

Compassion International has a “test and learn” program through which 10,000 sponsors each month are asked to make an additional gift. This provides opportunities to learn what motivates them to give by testing email copy. In June 2020, their team had drafted an email asking for a gift to the “Where Most Needed” fund, an unrestricted fund that allows Compassion to meet the most urgent needs of children. The original email made a direct ask

Research Question

How will adding specificity in impact and in the ask affect email conversion rate?

Design

C: Control
T1: Treatment #1

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.73%
T1: Treatment #1 1.5% 101.4% 100.0%

This experiment has a required sample size of 1,526 in order to be valid. Since the experiment had a total sample size of 20,000, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 101.4% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment, with increased specificity, increased response by 101.4%. This revealed a few powerful insights:

  1. People might have trouble wrapping their head around a problem expressed generally, or with statistics. This may make the problem seem too broad or difficult to solve. It also may be difficult for the donor to understand how their gift can make any difference at all.
  2. The Where Most Needed fund, which is a broad, unrestricted fund, seems to be most effectively communicated through the needs of the sponsor’s specific child. Explaining it in abstract might make it difficult to understand. But adding a person—and the extra layer of a person that the reader cares about—might have a multiplier effect.
  3. Helping a donor understand a specific goal that they can help reach—in this case, 17 respondents—gives them a very tangible thing to accomplish. Otherwise, they might rest on the assumption that someone else will do the job.


Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.