The Impact of Making Two CTAs in one Appeal Pt. 1
Care Net
Experiment Summary
Ended On: 12/10/2018
For months, Care Net has been brainstorming ways to increase the number of monthly donors, Caring Partners, that we have on file. We proposed testing our traditional appeal language against a new “Cut the Cord” campaign. This appeal would argue that money spent on entertainment providers like Netflix, Amazon Prime, Cable often supports pro-choice initiatives. By “cutting” some of our expenses on such entertainment and redirecting the money to Care Net’s work, lives would be saved and pro-choice orgs would lose funding. A win-win. We hoped that a pro-life person’s ire about pro-choice funding and programming would help motivate them to “cutting the cord” and saving lives.
The drawback to this approach would be that it would be essentially making two calls to actions: Cut expenses on entertainment and become a monthly donor.
Research Question
Would appealing to donors to “cut the cord” by cutting their entertainment bill (and its support of pro-choice programming) in favor of becoming a monthly donor increase the number of donations we received? What would the impact of making two CTAs in the same appeal be on donations?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.01% | ||
| T1: | Cut the Cord | 0.01% | -59.9% | 74.2% |
This experiment has a required sample size of 102,773 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
While the Cut the Cord email had the benefit of a new sender and a higher CTR, the donation rate was significantly lower. The results seemed to confirm that two CTAS in the appeal suppressed donor response as they had to agree to cutting back on something they were already doing and beginning something they weren’t yet doing. This was too much friction. However, we wanted to be sure. The variation had a 33% higher CTR. We wondered if we had too much copy on the donation page for the variation and that people dropped off who were prepared to make a gift. We decided to run an additional test to see if shorter copy on the donation page would cause Cut the Cord to perform better.