How personal email copy affects clickthrough rate Experiment ID: #3423

Jews for Jesus

Jews for Jesus exists to make the messiahship of Jesus an unavoidable issue to the Jewish people worldwide, utilizing creative evangelistic methods to reach the Jewish community and spread the good news of Jesus Christ.

Experiment Summary

Timeframe: 3/8/2016 - 3/14/2016

Jews for Jesus had a process in place to convert direct mail letters to eAppeals, which they sent on a monthly basis to their email file. While this raised money and kept digital donors engaged, they hypothesized that the tone of a direct mail piece wasn’t really aligning with the medium of email.

So while keeping the value proposition of the email intact, they completely rewrote it with a conversational tone. They also stripped all the elements of branding out of the email to make it seem like it came from a person, rather than email marketing software. Finally, they inserted personalization throughout the appeal.

Then, they split-tested this with their email file, dividing it in half to make sure they could validate a result.

Research Question

Will a personal tone with personalized copy increase clickthrough rate?

Design

C: Control
T1: Treatment 1

Results

Treatment Name Click Rate Relative Difference Confidence
C: Control 0.21%
T1: Treatment 1 0.44% 109.2% 100.0%

This experiment has a required sample size of 4,750 in order to be valid. Since the experiment had a total sample size of 119,721, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    109.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The new email produced a 109.2% increase in clickthrough rate. This provided insightful learnings about the Jews for Jesus digital donor:

  1. They respond when they feel personally engaged.
  2. They respond to emails that feel like they come from people, rather than email marketing machines.

This is difficult to do in direct mail, but the email provides many great ways to personalize communications. This points a way forward for all emails to be “high-touch”.


Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.