How prioritization combined with a direct ask affects conversion
Compassion International
Experiment Summary
Ended On: 02/26/2019
Compassion International, at any point, has thousands of children available for sponsorship. As users navigate throughout the site, the team had learned that they liked to sort and filter on certain variables to find the child they want to sponsor: boys vs girls, children in specific countries, ages, and so on. Many people used the search widgets to filter through children, because the number of children available was overwhelming—even after filtering through multiple searches. Their team had tested and found that prioritizing a single child on a birthday search results page produced a lift in conversion for that search, and wanted to see if the lift held on another search—longest waiting children. On the treatment, they put a single child at the top, and gave a direct appeal “Sponsor me” to the user. They hypothesized that this would force the user to consider one child instead of a grid of equally weighted children, and would increase conversion rates. They had to alter the message slightly to fit the search query, but were able to get a test up and running. This test ran after someone searched for “longest waiting children” as a filter, and was also linked from the navigation. They launched the test to determine a winner.
Research Question
How will prioritization and a direct ask affect conversion?
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Multiple Children Selected | 15.9% | ||
| T1: | Prioritized Child with Direct Ask | 17.6% | 11.2% | 95.2% |
This experiment has a required sample size of 3,405 in order to be valid. Since the experiment had a total sample size of 6,947, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 11.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
This shows the impact of “breaking the grid”—and also reveals something interesting about the sponsors: while we might think that they want more options to choose, they actually are more likely to choose a child when we let them take their hands off the wheel and choose for them. However, testing has also shown that they don’t want this to be entirely random, and they don’t want to feel like the child was selected for them. That’s why it’s important that they have some say in the matter—i.e., I want to see “longest waiting children”, but after that, they are more likely to sponsor the child prioritized and presented to them.