How copy length affects the value proposition of a Facebook ad Experiment ID: #6078

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 2/1/2017 - 2/8/2017

Hillsdale College launched new creative on Facebook promoting their course on the life of C.S. Lewis. Since these ads are direct response ads designed to attract viewers to sign up for the course, they had previously experienced much success with concise, clear copy that asked people to sign up.

However, they felt that they might have saturated the market of people who needed minimal persuasion to sign up, and hypothesized that to engage the next, less-motivated layer of the target audience, their ads might actually need more copy. They crafted copy for the top section of the ad that gave more insight into who C.S. Lewis was, and made a softer offer to sign up for the course.

They launched an A/B split test with the ads to determine a winner, knowing that Facebook would direct more traffic towards the ad that performed better.

Research Question

Will increasing the length and depth of the value proposition through the ad copy attract more people to click on the ad?

MECLABS Conversion Factors Targeted

C = 4m + 3v + 2( i - f) - 2a ©

Copyright 2015, MECLABS

Design

C: Shorter, direct copy
T1: Increased copy

Results

Treatment Name Click Rate Relative Difference Confidence
C: Shorter, direct copy 0.55%
T1: Increased copy 2.9% 426.1% 100.0%

This experiment has a required sample size of 315 in order to be valid. Since the experiment had a total sample size of 19,041, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    426.1% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The new copy produced a 421.6% increase in clickthrough rate. This seemed to validate that the target audience (at this point in the campaign) needed more copy to convince them to take the course, but there were a few other factors at play as well:

  • The new copy took up more vertical space, which captured a larger percentage of the Facebook newsfeed and drew the eye to the ad.
  • The increased length of the copy drew the eye to the ad and gave it the perception of having greater substance, which attracted more readers to engage with the copy.
  • The more inviting tone of the ad copy appealed to readers and drew them into the ad, rather than forcing them to make a quick decision about whether or not they wanted to sign up for a course that they didn’t know much about.

This learning provides immense opportunities for testing new ad creative across the board with all of Hillsdale’s courses, especially those that have run for an extended period of time.


Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.






Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.