How the cover image of an ebook increased results
The Fund for American Studies
Experiment Summary
Timeframe: 10/23/2019 - 11/08/2019
The Fund for American Studies launched a new acquisition offer and after running it for a month and a half, the team needed to improve ad clickthrough rate and email signup rates to boost performance for the offer.
Research Question
Would changing the ebook cover design image used in ads and on the landing page improve results?
Design
Results
Treatment Name | Conv. Rate | Relative Difference | Confidence | |
---|---|---|---|---|
C: | LFS - Cover Test (Control) | 1.9% | ||
T1: | LFS - Cover Test (Treatment) | 3.1% | 65.1% | 100.0% |
This experiment has a required sample size of 1,227 in order to be valid. Since the experiment had a total sample size of 21,678, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 65.1% increase in conversion rate
× 0% increase in average gift
Key Learnings
With a level of confidence of 99.9%, we validated an increase of 65.1% in email signup rate for this treatment.
This was partially driven by an increase of +25.9% in ad clickthrough rate (LoC: 99.2%). We also saw a directional lift in donor conversion rate by +6.4% (LoC: 9.8%).
The landing page results increased because the motivation of those arriving at the page, combined with the higher number of people (because more ads were clicked) increased results substantially.
Question about experiment #23941
If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.