How the cover image of an ebook increased results Experiment ID: #18699
The Fund for American Studies
Timeframe: 10/23/2019 - 11/8/2019
The Fund for American Studies launched a new acquisition offer and after running it for a month and a half, the team needed to improve ad clickthrough rate and email signup rates to boost performance for the offer.
Would changing the ebook cover design image used in ads and on the landing page improve results?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
|C:||LFS - Cover Test (Control)||1.9%|
|T1:||LFS - Cover Test (Treatment)||3.1%||65.1%||100.0%|
This experiment has a required sample size of 1,227 in order to be valid. Since the experiment had a total sample size of 21,678, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 65.1% increase in conversion rate
× 0% increase in average gift
With a level of confidence of 99.9%, we validated an increase of 65.1% in email signup rate for this treatment.
This was partially driven by an increase of +25.9% in ad clickthrough rate (LoC: 99.2%). We also saw a directional lift in donor conversion rate by +6.4% (LoC: 9.8%).
The landing page results increased because the motivation of those arriving at the page, combined with the higher number of people (because more ads were clicked) increased results substantially.