First Baptist Dallas

Does removing images and buttons and using plain-text on a high urgency campaign increase clicks for Pathway to Victory?

Experiment ID: #81622

First Baptist Dallas

Experiment Summary

Timeframe: 12/31/2021 - 12/31/2021

December 31st is usually the day when digital donors receive the most emails from nonprofits, as that is the day where the majority of them raise the most revenue. In order to cut through the ‘noise’ of December 31 emails, for Pathway to Victory we tested a plain-text email versus their control email which used graphics. The messaging remained the same on both versions; for our test however we made the email look like it came directly from Dr. Robert Jeffress. By removing all images, and removing hyperlinked text, while adding in the full URLs for the landing page, our goal was to increase the number of click throughs.

Research Question

We believe that removing images for email subscribers will achieve a higher click through rate.

Design

C: Control
T1: Plain Text Email

Results

  Treatment Name Click Rate Relative Difference Confidence
C: Control 0.80%
T1: Plain Text Email 1.7% 110.3% 100.0%

This experiment has a required sample size of 953 in order to be valid. Since the experiment had a total sample size of 60,114, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    110.3% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

By making the digital donor feel like they were being personally reached out to by Dr. Jeffress, instead of receiving a generic ‘marketing’ email with graphics, we drove a higher click through rate and engagement with Pathway to Victory. This hypothesis will be useful to continue to test moving forward with not just high-urgency campaigns like a December 31st email, but throughout the year.


Experiment Documented by NextAfter

Question about experiment #81622

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.