Alliance Defending Freedom

How taking a softer call to action affects donations

Experiment ID: #1345

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 12/26/2014 - 12/27/2014

As it was getting closer to the end of the year, Heritage wanted to test a messaging approach with their fifth email in their year end campaign. Toward the end of urgency-based marketing campaigns, it is common practice to make a direct ask in the email. The question was how early can that direct ask be made and have it still be effective.

Research Question

A softer call-to-action within the email will drive more traffic to the website which will, in turn, convert a greater number of donors.


C: Direct Ask
T1: Soft Call-to-Action


  Treatment Name Conv. Rate Relative Difference Confidence
C: Direct Ask 0.03%
T1: Soft Call-to-Action 0.03% -4.3% 11.6%

This experiment has a required sample size of 15,418,569 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

The softer language in the email increased the number of visitors by 41% but it actually converted fewer total donors in the test. At this stage in the campaign, there does not seem to be any impact to response rate by utilizing a softer call to action.

Experiment Documented by NextAfter

Question about experiment #1345

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.