How adding value proposition language to a direct appeal email impacted donations.
Illinois Policy Institute
Experiment Summary
Timeframe: 12/28/2022 - 12/30/2022
As part of a high-urgency campaign for a public policy client, we emailed subscribers to their weekly newsletter, The Policy Shop, asking if they would make a donation in support of The Policy Shop. The email was short and sweet, including a cartoon, a thanks for being a part of The Policy Shop, and an ask for support.
We wondered how a more in-depth, value proposition-focused email would impact donations. The treatment version highlights the problem(s) that The Policy Shop helps solve, provides background on The Policy Shop, provides evidentials on the impact of The Policy Shop, and tells the recipients how their gift in support of The Policy Shop will be used. The treatment also takes a more personal approach by greeting the recipient by name and including a raw donation link instead of a button and graphic.
Research Question
We believe that a value proposition-enhanced email for email recipients will achieve an increase in donations.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.00% | ||
| T1: | Treatment #1 | 1.3% | 100.0% | 98.6% |
This experiment has a required sample size of 301 in order to be valid. Since the experiment had a total sample size of 970, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 100.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment resulted in a valid 1.3% increase in donations, with six coming from the treatment and none coming from the control. Although the treatment produced fewer clicks, it did produce a far better donor conversion rate, which is more valuable. This proves that the value proposition-heavy email was more valuable. Those who clicked the treatment email were more aware of the impact The Policy Shop has and why it needs support to continue to have such an impact. They had greater clarity on why The Policy Shop is so important. This clarity and motivation gained from the value proposition in the treatment could not be gained from the control. The experiment stresses the importance of value proposition copy and proves that value proposition should be a strong focus in emails going forward.