How adding first name personalization to a donation page affected donor conversions
Reasons to Believe
Experiment Summary
Timeframe: 06/09/2022 - 07/30/2022
Reasons to Believe was running a survey offer that directed users to an instant donation page after completion. We wanted to see an increase in the instant donor conversion rate. We hypothesized that by personalizing the headline with a first name it would create a greater connection with the reader and communicate a deeper appreciation for their participation. Additionally, we suspected that the control headline was communicating to the user that there was no further action to be taken: “Thank you for your insights and for giving your opinion.” Without changing the primary message, we believed using a personalization would make for a headline that is less likely to be ignored.
Research Question
We believe that by personalizing the headline with a first name for survey participants will achieve an increase in donor conversion rate..
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.91% | $0.00 | ||
| T1: | First name personalization | 2.0% | 124.0% | 83.6% | $0.00 |
This experiment has a required sample size of 874 in order to be valid. Since the experiment had a total sample size of 878, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
While the level of confidence still encourages room for testing, there are a few notable hints in the data regarding our initial concept. Our hypothesis was that a name personalization would show greater appreciation and stand out to the user. One interesting comparison is that the treatment caused little to no change in conversion rate for new visitors, but did cause a slight increase for returning visitors. But alternatively, Organic Search visitors had a 67% increase in donations with a 99% level of confidence. Is there a learning to be had on when users perceive the most value in personalization? Is there thought to be more value in someone “remembering” our name, than an organization using it directly after it was provided?
Ultimately, more testing is still required in order to validate these findings.
- Revenue experienced a 85.0% decrease with a 93% level of confidence.
Observations of the impact on other visitor segments in the experiment:
- Desktop visitors had a 17.1% increase in donations with a 99% level of confidence.
- Male visitors had a 9.7% increase in donations with a 99% level of confidence.
- 55-64 visitors had a 14.7% increase in donations with a 98% level of confidence.
- Organic Search visitors had a 66.7% increase in donations with a 99% level of confidence.