How removing branding and social proof to simplify a petition impacted donor conversion rate
Canadian Taxpayers Federation
A not-for-profit citizen's advocacy group dedicated to lower taxes, less waste and accountable government.
Experiment Summary
Ended On: 08/20/2020
As a part of optimizing petitions used for the acquisition of new activists and donors, the Canadian Taxpayer Federation looked to experiment with the branding, which also included a social proof element, from the petition signing page. How would this impact email signup rate, and downstream donor and revenue metrics?
Research Question
Does removing branding (and the corresponding social proof) impact email signup, donor conversion or revenue rates?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.23% | ||
| T1: | Treatment #1 | 0.08% | -65.7% | 98.3% |
This experiment has a required sample size of 5,133 in order to be valid. Since the experiment had a total sample size of 15,135, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 65.7% decrease in conversion rate
× 0% increase in average gift
Key Learnings
With a 98.3% level of confidence, we observed a decrease in instant donor conversion rate of 65.7% when the organization’s logo and the corresponding social proof was removed from the top of the petition signature page.
There are two other interesting observations, which are:
- The email signup rate slightly increased — showing a +2.8% (LoC: 96.7%), but the decrease in subsequent donor conversion rate also drove a …
- Decrease in revenue of -60.6% (LoC: 95.4%).
Could the organization’s branding and social proof (showing the size of the community they would be joining if they made a gift on the subsequent donation page after signing the petition) decrease their likelihood to give a gift at all?
Ultimately, we decided to turn off the experiment because the increase in email signup rates did not jump high enough to offset the big decreases we witnessed in donor conversion and revenue.
Further experimentation may be required, but it certainly appears that when you have social proof to publicize, that it is far more important to publicize it all of the way through the funnel instead of removing it, as it appears to have negatively impacted subsequent giving.