How the design of a fundraising email affects clickthrough rate

Experiment ID: #4174


CaringBridge offers free personal, protected websites for people to easily share updates and receive support and encouragement from their community during a health journey. Every 7 minutes, a CaringBridge website is created for someone experiencing a health event.

Experiment Summary

Timeframe: 03/02/2016 - 03/30/2016

CaringBridge received a matching gift for their spring fundraising campaign from a board member. Since the majority of their audience was unfamiliar with the board member, Rik Lalim, they originally created a templated email that was explicitly “CaringBridge” to ease users’ potential anxiety. But they had previously tested emails that had no formatting and emulated “normal” email, and experienced great success. They decided to create a treatment version that stripped away all the formatting and photography.

They then split their email file and ran a test to see which attracted more responses

Research Question

Will a stripped-down fundraising email increase clickthrough rate?


C: Template design
T1: Personal Email


 Treatment NameClick RateRelative DifferenceConfidence
C: Template design 0.17%
T1: Personal Email 0.30%80.3% 100.0%

This experiment has a required sample size of 9,913 in order to be valid. Since the experiment had a total sample size of 210,545, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    80.3% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The stripped-down email (with no template) produced an 80% increase in clickthrough rate. This tells us something about trust — and giving is an act of trust. It may imply that people are more likely to take action when they feel that the email is coming from a person, rather than an organization.

There were still elements of the treatment email that weren’t entirely personal — the big blue button, and the logo. This prompted further testing to see if the removal of these elements produced an additional lift.

Experiment Documented by Jeff Giddens
Jeff Giddens is President of NextAfter.

Question about experiment #4174

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.