How increasing the clarity about the author impacted email signup rates for an eBook offer
The Fund for American Studies
Experiment Summary
Timeframe: 08/25/2020 - 09/03/2020
As a part of the launch of a new eBook, The Fund for American Studies decided to experiment with increasing author clarity for the new resource they were promoting. Could this increase email signup rates, and subsequent donor conversion rates?
Research Question
How does increasing the author clarity of a new eBook offer impact email signup and donor conversion rates?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | No Author Clarity | 61.4% | ||
T1: | w/Author Clarity | 53.0% | -13.7% | 100.0% |
This experiment has a required sample size of 266 in order to be valid. Since the experiment had a total sample size of 2,105, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 13.7% decrease in conversion rate
× 0% increase in average gift
Key Learnings
With a 100% level of confidence, we observed a decrease in email signup rate of -13.7% with the increased clarity for who the author is.
Subsequently, we not only observed fewer downloaders going through the treatment page, but with an 80% level of confidence (thereby making the observation not valid—yet, directionally accurate) we also observed a decrease in subsequent donor conversion by 29.9%, as well.
This is a baffling study. Typically, increasing the clarity of some component of the offer (in this case—who wrote it) will increase the strength of the value proposition, and subsequently increase results.
For that reason, we recommend further experimentation with this concept in terms of layout/design to continue testing this hypothesis to see if we can separate the author clarity from the presentation of the information itself.
Question about experiment #35438
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.