How a text ad affects conversion for a free trial offer signup Experiment ID: #9358
Ended On: 8/21/2018
National Catholic Register had run many animated banner ads throughout their site to encourage online readers to sign up for a free three-month trial. While the animated banners were eye-catching and featured well-known Catholic personalities, they wondered if they required the viewer to sit for a long period in front of the ad to see the payoff—and if those viewers, once the reveal was complete, missed the CTA entirely. Also, they wondered if the narrative form of the ad was less clear and straightforward than a simple text offer with a bold call-to-action.
They created a treatment ad, run through their ad server, that presented the trial offer in a clear way with a big CTA button. They evenly apportioned the ads to run to NCRegister viewers and closely tracked metrics to determine clicks and conversions.
How will a text-only ad perform when compared to an animated banner?
|Treatment Name||Conv. Rate||Relative Difference||Confidence|
This experiment has a required sample size of 42,415 in order to be valid. Since the experiment had a total sample size of 310,791, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 296.0% increase in conversion rate
× 0% increase in average gift
The text ad produced a 296% lift in overall conversions, from impressions to clicks, nearly quadrupling the number of conversions from the animated banner. Interestingly, the text banner reduced clickthrough rate by 18.5%, but increased follow-on conversion by 386%. It was hypothesized that the primary factor in the difference in clicks was due to the much larger clickable area on the animated banner, which, on mobile, was much more susceptible to accidental clicks. However, the conversion rate from the text ad was so much higher that some portion of this conversion increase can also be attributed to the increased clarity of the offer—greater understanding from the end user resulted in a “more motivated click”.
This produced future test ideas, like adding a picture of the print edition, and specifically calling the offer a print edition.