How adding detail of an adversity in a landing page affects conversion Experiment ID: #11664

Leadership Institute

Experiment Summary

Ended On: 2/7/2019

The Leadership Institute ran a campaign around stopping the left’s push for Trump’s impeachment. They wanted to know if having shorter copy that briefly stated how to confront and fix the adversity (radical leftist professors teaching students how to get Trump impeached) would result in more conversions in comparison to longer copy that includes more detail about how to confront and fix adversity.

Research Question

Do people need more detail on how to fix a problem before they feel confident enough to give?

Design

C: Control
T1: Treatment #1

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Control 2.3%
T1: Treatment #1 3.5% 51.9% 99.3%

This experiment has a required sample size of 1,514 in order to be valid. Since the experiment had a total sample size of 5,775, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 51.9% increase in conversion rate
× 0% increase in average gift

Key Learnings

The short copy resulted in fewer donations than the longer copy. With a 51.9% increase in average gift, the Leadership Institute can include that long copy is okay to use as long as it gives more detail on how the potential donor’s donation can stop the problem being faced.


Experiment Documented by...

Kevin Peters

Kevin is the Chief Technology Officer at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.