How “shared responsibility” affects clickthrough rate on a Facebook ad
Texas Public Policy Foundation
Experiment Summary
Ended On: 04/12/2019
Texas Public Policy Foundation was promoting a property tax survey, which was followed by a donation ask. The original version of the Facebook ad asked the reader to “join your fellow Texans” and take the survey. However, they wondered if this “shared responsibility” might actually be negatively affecting clickthrough rate. The reader might think that there are plenty of other people taking the survey, and not feel inclined to complete it personally. They removed that line and launched a test to see if they could improve response rate.
Research Question
How will removing “shared responsibility” to highlight personal responsibility affect response rate?
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Shared responsibility | 2.3% | ||
| T1: | Personal responsibility | 2.9% | 25.0% | 99.4% |
This experiment has a required sample size of 6,231 in order to be valid. Since the experiment had a total sample size of 47,725, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
25.0% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The second ad, which removed the “shared responsibility” sentence from the copy produced a 25% increase in clickthrough rate, which led to a large increase in both names acquired and donations. A single sentence might not presume to have so much impact, but considering the audience, personal responsibility is a big motivator. This learning can be used in future offers to optimize results.