How emphasizing data security on a petition impacted donor conversion rate
Canadian Taxpayers Federation
A not-for-profit citizen's advocacy group dedicated to lower taxes, less waste and accountable government.
Experiment Summary
Timeframe: 04/22/2022 - 05/30/2022
As a part of continuous testing, the Canadian Taxpayer Federation wanted to try a simple message to emphasize data security on a petition to see how it would impact their email signup rate, donor conversion rate, and subsequently revenue.
Research Question
We believe that emphasizing data security for prospects signing a petition will achieve increase email signup rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 0.28% | $0.00 | ||
| T1: | w/Data Security Message | 0.58% | 102.9% | 99.0% | $0.00 |
This experiment has a required sample size of 3,836 in order to be valid. Since the experiment had a total sample size of 13,269, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 102.9% increase in conversion rate
× 0% increase in average gift
Key Learnings
Much to our surprise, the treatment actually decreased email signup rates by -2.1% (LoC: 99%), but while we saw fewer email signups for prospects who were segmented into the treatment experience, we did see an increase in donor conversion rate of +102.9% (LoC: 98.9%) on the subsequent instant donation page.
We also observed an increase in revenue of +189.5% (with an LoC of 100%), as well.
The real question is: Why did this happen?
This experiment actually lines up with other experiments that increase security at the email signup rate that we’ve seen, where the email signup rate decreases slightly in exchange for a much higher donor conversion rate on the subsequent instant donation page.
The hypothesis as to why the email signup rate decreases is that it draws some attention to a prospect who otherwise wouldn’t think that it was safe to provide their email address and cause a small percentage to reconsider the offer altogether.
Meanwhile, those that decide to proceed to feel safer in the decision to make a gift on the subsequent instant donation page, because they already know that you take the safety and security of their personal information seriously, and it’s through this increase in trust that they’ll decide to give at a higher rate.
This result really just simply comes down to the principle of “decreasing anxiety” (which is defined as “a psychological to take the presented action“) within the mind of the prospective donor by letting them know that they can trust you.
Our upcoming experiment with this treatment is to increase the visual appearance of security on the email signup form along with the disclaimer language designed to heighten the perception of security to even greater levels.