How a different messaging angle affects conversion rate from a Facebook ad
NextAfter
Experiment Summary
Timeframe: 01/01/2020 - 01/08/2020
At the start of the year, we launched a new Facebook ad campaign promoting the new State of Nonprofit Email Cultivation research report. As this is the first time to push the report in a focused way, we want to test a couple different messaging angles that each seemed to be a viable approach.
The control copy focused in on known stats about donor retention in order to a make the argument that something is broken with our cultivation strategies.
The treatment copy focused on the donor’s experience, illustrating how frustrated and jaded donors can feel if they are not cultivated well.
Each version ended up making the same call-to-action, using the same headline, image, etc. The primary change was the first 2-3 paragraphs of copy to set up the call-to-action.
Research Question
Which value proposition angle will lead to more report downloads?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | "Something is Broken" Value Proposition | 0.40% | ||
| T1: | "Frustrated and Jaded Donors" Value Proposition | 1.1% | 168.9% | 100.0% |
This experiment has a required sample size of 1,398 in order to be valid. Since the experiment had a total sample size of 7,911, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 168.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
From this experiment, we learned that the “frustrated and jaded donors” approach is a more effective messaging approach – to the tune of a 168% increase in conversion over the control. It is interesting to see that data alone does not always move people to action. But using copy to make a more personal and human connection – and helping fundraisers to empathize with donors – can be a stronger copywriting method.
We’ll need to put this broader learning to the test in other scenarios to see if it holds true.