Heritage Action for America

Email invoking disgust and sadness vs. Email highlighting betrayal of GOP leadership

Experiment ID: #2709

Heritage Action for America

Experiment Summary

Timeframe: 07/31/2015 - 07/31/2015

We have evidence through market research and focus groups that our donors are fed up with GOP leadership. We are looking for ways to show that supporting Heritage Action is the way to get things done in Washington. We are the solution to bypassing all of the Republican Establishment tricks and getting conservative legislation signed into law.

Research Question

Which would drive more gifts and revenue: an email that invokes the emotion of horror and sadness in the response to the disturbing undercover Planned Parenthood videos or an email that shows the betrayal of GOP leadership to defunding Planned Parenthood?



C: Disgust
T1: Betrayal of GOP Leadeship


 Treatment NameConv. RateRelative DifferenceConfidence
C: Disgust 0.04%
T1: Betrayal of GOP Leadeship 0.12%191.8% 100.0%

This experiment has a required sample size of 10,007 in order to be valid. Since the experiment had a total sample size of 117,471, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 191.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

The email that highlighted the betrayal of GOP leadership won. When roll-out to the rest of the file it secured an additional $8,392 from 208 gifts with an average gift of $40.

We learned that connecting what GOP leaders say with what they actually do helps motivate donor to give to the solution.

Experiment Documented by NextAfter

Question about experiment #2709

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.