Does removing images and buttons and using plain-text on a high urgency campaign increase clicks for Pathway to Victory?
First Baptist Dallas
Experiment Summary
Timeframe: 12/31/2021 - 12/31/2021
December 31st is usually the day when digital donors receive the most emails from nonprofits, as that is the day where the majority of them raise the most revenue. In order to cut through the ‘noise’ of December 31 emails, for Pathway to Victory we tested a plain-text email versus their control email which used graphics. The messaging remained the same on both versions; for our test however we made the email look like it came directly from Dr. Robert Jeffress. By removing all images, and removing hyperlinked text, while adding in the full URLs for the landing page, our goal was to increase the number of click throughs.
Research Question
We believe that removing images for email subscribers will achieve a higher click through rate.
Design
Results
Treatment Name | Click Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | Control | 0.80% | ||
T1: | Plain Text Email | 1.7% | 110.3% | 100.0% |
This experiment has a required sample size of 953 in order to be valid. Since the experiment had a total sample size of 60,114, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
110.3% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
By making the digital donor feel like they were being personally reached out to by Dr. Jeffress, instead of receiving a generic ‘marketing’ email with graphics, we drove a higher click through rate and engagement with Pathway to Victory. This hypothesis will be useful to continue to test moving forward with not just high-urgency campaigns like a December 31st email, but throughout the year.
Question about experiment #81622
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.