How focusing on a movement instead of the opposition affects donor conversion Experiment ID: #14365

Leadership Institute

Experiment Summary

Ended On: 4/16/2019

Leadership Institute was running a campaign to start a conservative group on every college campus. They had an ad that focused on the opposition that young conservatives face on campus. However, they wondered if their audience was weary of seeing the opposition, and instead might be motivated by seeing students who were trying to make a difference. They created an ad that didn’t start with a the opposition, but started with a problem that the donor could actually hope to solve. They used specific numbers to help quantify the goal, and then they used a picture of conservative students rather than opposition protest groups. They launched an A/B test to determine a winner.

Research Question

How will a positive, movement-focused ad perform against an opposition focused ad?

Design

C: Opposition focused
T1: Movement focused

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Opposition focused 0.02%
T1: Movement focused 0.08% 298.1% 98.5%

This experiment has a required sample size of 12,855 in order to be valid. Since the experiment had a total sample size of 29,941, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 298.1% increase in conversion rate
× 0% increase in average gift

Key Learnings

The positive, movement-focused ad produced a 298.1% increase in conversion rate—which seemed to indicate that donors are more motivated as to what they can do, as opposed to what they are up against. This might help green-light solutions-focused offers instead of trying to activate protest gifts.


Experiment Documented by...

Jeff Giddens

Jeff is the President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.