The Heritage Foundation

How text reminders impacted campaign giving

Experiment ID: #8932

The Heritage Foundation

Founded in 1973, The Heritage Foundation is a research and educational institution—a think tank—whose mission is to formulate and promote conservative public policies based on the principles of free enterprise, limited government, individual freedom, traditional American values, and a strong national defense.

Experiment Summary

Timeframe: 04/17/2018 - 04/24/2018

For the 2018 Tax Day Moneybomb, we decided to test out a next texting technology to see if reminding donors to give on the day of a big campaign would increase overall conversion. Since we did not have a significant volume of cell phone numbers, we had to append the numbers to donor records. We began the campaign with a cultivation text the day before the deadline and then a reminder to give on the day of the deadline.

Research Question

Will cultivating donors via text during a moneybomb campaign increase overall conversion?

Design

C: No Texts Sent
T1: Received Texts

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: No Texts Sent 0.07%
T1: Received Texts 0.09%23.1% 28.7%

This experiment has a required sample size of 208,124 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

When we compared those donors that were sent a text with those that could have received a text, we see that there was no statistically significant difference between the two segments. When we also factor in the time it took to set up the texting, the complaints received from members, and relatively low potential impact, this does not seem like a tactic that would be worth repeating for future campaigns.


Experiment Documented by NextAfter

Question about experiment #8932

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.