Canadian Taxpayers Federation

How removing branding and social proof to simplify a petition impacted donor conversion rate

Canadian Taxpayers Federation

A not-for-profit citizen's advocacy group dedicated to lower taxes, less waste and accountable government.

Experiment Summary

Ended On: 8/20/2020

As a part of optimizing petitions used for the acquisition of new activists and donors, the Canadian Taxpayer Federation looked to experiment with the branding, which also included a social proof element, from the petition signing page. How would this impact email signup rate, and downstream donor and revenue metrics?

Research Question

Does removing branding (and the corresponding social proof) impact email signup, donor conversion or revenue rates?

Design

C: Control
T1: Treatment #1

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: 0.23%
T1: Treatment #1 0.08% -65.7% 98.3%

This experiment has a required sample size of 5,133 in order to be valid. Since the experiment had a total sample size of 15,135, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 65.7% decrease in conversion rate
× 0% increase in average gift

Key Learnings

With a 98.3% level of confidence, we observed a decrease in instant donor conversion rate of 65.7% when the organization’s logo and the corresponding social proof was removed from the top of the petition signature page.

There are two other interesting observations, which are:

  1. The email signup rate slightly increased — showing a +2.8% (LoC: 96.7%), but the decrease in subsequent donor conversion rate also drove a …
  2. Decrease in revenue of -60.6% (LoC: 95.4%).

Could the organization’s branding and social proof (showing the size of the community they would be joining if they made a gift on the subsequent donation page after signing the petition) decrease their likelihood to give a gift at all?

Ultimately, we decided to turn off the experiment because the increase in email signup rates did not jump high enough to offset the big decreases we witnessed in donor conversion and revenue.

Further experimentation may be required, but it certainly appears that when you have social proof to publicize, that it is far more important to publicize it all of the way through the funnel instead of removing it, as it appears to have negatively impacted subsequent giving.


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #9762

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.