How the length of a pledge impacts email conversion rate
Ended On: 9/15/2020
In an effort to acquire email addresses, KCBI created a pledge/petition for people to sign that aligned people’s beliefs with a core value of theirs. The original pledge had ten statements of belief within it and we wondered if the number of beliefs statements would impact conversion.
Ultimately what we were trying to figure out is this, can a pledge have too many belief statements resulting in friction and therefore drop-offs?
We cut the number of belief statements from 10 to 5 for the treatment. We chose the 5 for the treatment based on what we thought were the most “consumable” for people – they weren’t lengthy, weighted, and seemed easiest to understand, read, and align with.
Does the length of a pledge/petition have a difference on the conversion rate?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 346 in order to be valid. Since the experiment had a total sample size of 740, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 9.3% increase in conversion rate
× 0% increase in average gift
What we ended up seeing was a 9.3% increase in conversion for the shorter pledge! Our biggest takeaway here was that less is sometimes more. The pledge with 5 belief statements converted at a higher rate than the one with 10. This could be based on a couple of things…
1. The length of the pledge
2. Simplicity of the statements of beliefThe next step would be to look at the language and wording of each statement and begin to test there.
Now that we have seen that less is more we can begin iterating on what words or phrasing should be used.
Question about experiment #9794
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.