Alliance Defending Freedom

How emphasizing the urgency of a campaign impacts donor conversion

Experiment ID: #8028

Alliance Defending Freedom

Alliance Defending Freedom is an alliance-building legal organization that advocates for the right of people to freely live out their faith.

Experiment Summary

Timeframe: 10/17/2017 - 11/30/2017

Alliance Defending Freedom was looking for new ways to optimize the performance of their Jack Phillips donation page. They wondered: Can we positively impact visitor motivation and by increasing the relevance and urgency donating to support the Jack Phillip’s case? They created a treatment version and launched an A/B test to find out.

Research Question

Will donation page copy that emphasizes the relevance and urgency of Jack’s case increase donor acquisition?

Design

C: Control
T1: Treatment 1

Results

 Treatment NameConv. RateRelative DifferenceConfidence
C: Control 43.8%
T1: Treatment 1 42.1%-4.0% 35.0%

This experiment has a required sample size of 6,084 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

Alliance Defending Freedom found that this test produced no statistically significant results. Therefore, the impact of the treatment version is inconclusive. Still, the fact that this page did not produce a valid result does tell us that the copy used on the treatment page to emphasize the urgency of this case did not increase visitor motivation.


Experiment Documented by NextAfter

Question about experiment #8028

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.