How adding a person’s name to the confirmation page of a pledge impacts donor conversion
Live Action
Experiment Summary
Timeframe: 08/20/2020 - 11/16/2020
Live Action uses a pledge as an email acquisition opportunity. Through recent experiments with other organizations, we found that by personalizing the instant donation page that people are taken to after they sign a pledge can increase donor conversion. To test this with Live Action, we simply added the person’s first name to the headline of the confirmation/instant donation page. This data was passed through the URL from the form field on the pledge.
Research Question
Will personalizing the instant donation page increase donor conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.2% | ||
| T1: | Name personalization | 1.0% | -14.3% | 89.8% |
This experiment has a required sample size of 28,404 in order to be valid. Since the experiment had a total sample size of 39,615, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
We weren’t able to validate this experiment, but we did see a directional 14% decrease in donor conversion. We saw this similar result across all channels, device types, and user types. Our hypothesis with seeing a decrease in donor conversion when using this personalization tactic is that instead of increasing motivation we increased friction on the page. Maybe the use of their first name was too invasive for the prospective donors.
We’d like to continue to test into this theory around personalization as we’ve seen it perform consistently well with other organization and with similar offers.