How using donor-centric language in a calendar year end appeal impacted clicks
Dallas Theological Seminary
The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.
Experiment Summary
Timeframe: 12/08/2021 - 12/08/2021
As we reviewed copy for the year end campaign, we realized the direct mail messaging for the campaign was very organization-centric and not at all focused on the donor or their impact. A single letter referenced the organization “DTS” 12 times and contained the word “We” another 10 times while only using the word “you” twice. We decided to test how this message would compare to a more personalized appeal that used donor-centric language. The treatment presented the same information and value proposition for why a donor should give, but it was heavily revised to focus on the donor rather than the accomplishments of the organization. The treatment copy reduced “DTS” references to just four mentions and used one-to-one communication including the word “I” four times and “you/your” 10 times.
Research Question
We believe that using donor-centric language for a calendar year end appeal email will achieve a higher click-through rate.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.06% | ||
| T1: | Treatment #1 | 0.11% | 85.9% | 99.2% |
This experiment has a required sample size of 24,973 in order to be valid. Since the experiment had a total sample size of 90,563, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
85.9% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
We actually had the opportunity to test this concept twice. The first experiment only reached 45% level of confidence that there was any lift in traffic, but on the second send to recipients who had not opened the first email, we saw an 85% increase in traffic to the donation page with 99% confidence. The donor-centric language motivated more visitors to take the next step toward making a gift. We will want to test this approach again during the next high urgency campaign. Once they arrived on the donation page, we measured instant conversion rate there as well and did not achieve a valid level of confidence that there was any lift in donations from the segment who arrived on the page from the treatment version of the email. The learnings from this test could be applied and tested in the next direct mail appeal as well to see how donor-centric language performs in another channel.