How length and presentation of email copy impact donor conversion rate | NextAfter
Leadership Institute

How length and presentation of email copy impact donor conversion rate

Experiment ID: #8899

Leadership Institute

Experiment Summary

Timeframe: 04/10/2018 - 04/25/2018

Leadership Institute was sending their monthly appeal to their email file, based on the direct mail letter that also went out during that month. They were curious to see how including the full text (including extra spacing) would perform as compared to a truncated email version of the copy that distilled the value proposition into a shorter length email.

They created both versions and split the housefile to test.

Research Question

How will length and presentation of email copy impact conversion rate?


C: Control
T1: Treatment #1


  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 27.5%
T1: Treatment #1 11.6% -57.7% 99.6%

This experiment has a required sample size of 42 in order to be valid. Since the experiment had a total sample size of 269, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 57.7% decrease in conversion rate
× 0% increase in average gift

Key Learnings

Though this experiment produced a big lift in the clickthrough rate, it resulted in a conversion rate that was half of the control—meaning that “more clicks” did not result in more donors. This shows the power of copy to motivate email readers to action—more copy produced fewer, more motivated clicks.

This suggests a future test to adjust the presentation of the email without shortening the copy.

Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #8899

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.