How an ipad image versus a book image affects email conversion rates for an eBook offer
The Missionary Oblates of Mary Immaculate
The Missionary Oblates of Mary Immaculate are a Roman Catholic congregation of priests and brothers founded after the French Revolution by St. Eugene De Mazenod to work among the poor. Today there are nearly 4,000 missionaries working in more than 60 countries around the world.
Experiment Summary
Ended On: 02/10/2021
The Oblates switched from a book image to an iPad image to show their constituents the accurate medium for the offer. Especially as they were receiving complaints from their constituents that they did not get the actual book shown in the image. However, this change was made without testing. So we wanted to ensure that using an iPad image instead of a book image did not hurt conversion.
Research Question
Using an iPad image may decrease perceived value and so decrease conversion rates.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | iPad image | 42.4% | ||
| T1: | Book Image | 71.9% | 69.5% | 100.0% |
This experiment has a required sample size of 22 in order to be valid. Since the experiment had a total sample size of 2,089, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 69.5% increase in conversion rate
× 0% increase in average gift
Key Learnings
The book image increased email acquisition rate on the landing page by 70%. But the click through rate only saw a direction lift of 12% increase, that had a 77% level of confidence. The donation conversion rate saw no meaningful changes.