Email invoking disgust and sadness vs. Email highlighting betrayal of GOP leadership
Heritage Action for America
Experiment Summary
Timeframe: 07/31/2015 - 07/31/2015
We have evidence through market research and focus groups that our donors are fed up with GOP leadership. We are looking for ways to show that supporting Heritage Action is the way to get things done in Washington. We are the solution to bypassing all of the Republican Establishment tricks and getting conservative legislation signed into law.
Research Question
Which would drive more gifts and revenue: an email that invokes the emotion of horror and sadness in the response to the disturbing undercover Planned Parenthood videos or an email that shows the betrayal of GOP leadership to defunding Planned Parenthood?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Disgust | 0.04% | ||
| T1: | Betrayal of GOP Leadeship | 0.12% | 191.8% | 100.0% |
This experiment has a required sample size of 10,007 in order to be valid. Since the experiment had a total sample size of 117,471, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 191.8% increase in conversion rate
× 0% increase in average gift
Key Learnings
The email that highlighted the betrayal of GOP leadership won. When roll-out to the rest of the file it secured an additional $8,392 from 208 gifts with an average gift of $40.
We learned that connecting what GOP leaders say with what they actually do helps motivate donor to give to the solution.