How clarifying the value proposition affects clickthrough rate
Compassion International
Experiment Summary
Ended On: 03/04/2021
Compassion International was running ad to drive traffic to a campaign that offered users the ability to round up their purchases and donate the marginal “change”. The original campaign copy started with a high-level value proposition—”Poverty is complex. Generosity can be simple”. Then, it offered the user the chance to join millions of others who are donating change, and then explained the concept of RoundUp without naming the brand.
The team hypothesized that clarifying the value proposition by:
- creating an ad that spoke more directly to the user on a 1:1 basis,
- AND informed them of the impact they could personally make,
- AND connected the impact to a specific child, rather than a cause,
- along with a “preview” of the brand name (RoundUp)
…would attract more clicks. They decided to test this with an ad in Facebook.
Research Question
We believe that clarifying the value proposition for Facebook users will achieve an increase in clickthrough rate.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.18% | ||
| T1: | Treatment #1 | 0.43% | 134.2% | 100.0% |
This experiment has a required sample size of 3,377 in order to be valid. Since the experiment had a total sample size of 159,307, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
134.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment ad produced a 134.2% increase in clickthrough rate—which showed that the clarified value proposition was more likely to attract a click than the control ad.
However—since all clicks are not created equal, downstream conversions (initial signups and “added credit cards”) must also be validated to make sure that these clicks are equally motivated as the control group.