How a new value proposition focus in a sales email affects mystery donor audit applications
NextAfter
Experiment Summary
Timeframe: 12/13/2018 - 12/20/2018
In this experiment, we wondered if a different value proposition approach to a mystery donor audit sales email would lead to greater applications. The control we had tested into already focused on how expert opinions are really just that : opinions. If your fundraising isn’t based on research, data, and testing, then you’re just relying on intuition.
The treatment focused on the lack of time to the necessary research to create a plan for 2019. We could step in with this mystery donor audit and do the leg work for you to help you create a strategy for your next year of fundraising. Both the subject line and body copy of the email were included in the treatment variables.
Research Question
Will focusing the value proposition on the lack of time to plan and research lead to greater applications?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | What are your 2019 online fundraising plans? | 1.4% | ||
| T1: | Here’s a great way to increase giving for 2019 and beyond | 0.42% | -70.0% | 89.6% |
This experiment has a required sample size of 734 in order to be valid. Since the experiment had a total sample size of 980, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
The treatment version saw a 70% decrease in clicks through to the landing page. While we didn’t reach full statistical confidence, we were approaching confidence at 89.5%. While we didn’t have enough applications to validate off of conversions (3 for the control and 1 for the treatment), this decrease in clicks is a strong indicator that focusing on time constraints is not the right value proposition focus.