How strengthening the value proposition of a homepage donation CTA impacts clicks
CBN
Experiment Summary
Ended On: 08/17/2023
CBN is a Christian news, media, and distribution organization with a mission to preach the Gospel and demonstrate God’s love for the people of the world. The homepage of their website features an image with a Call-to-Action (CTA) that encourages readers to make a gift to CBN and support their work to share Christ’s love with the world.
For this test, we wanted to test into the impact of enhancing the value proposition language around this CTA by adding additional clarity, appeal, and credibility. We ran this test with the following variants:
- Control: This variant featured a short and simple headline that said “Sharing the Love of Christ Around the World” above the “GIVE NOW” button
- Treatment #1: This variant was designed to give readers a better understanding of how CBN shares Christ’s love by explaining that the reader’s gift to CBN will provide humanitarian aid and the Gospel to people. The headline in this variant said “Your gift right now will reach people with humanitarian aid and the Gospel of Jesus”
- Treatment #2: Like Treatment #1, this variant added clarity to the headline, but it also added body copy that explained the problem more, starting with “Millions of people still lack access to the Gospel, hot food, and clean water but you can help change that.” It also reassured readers that 100% of their donation would support CBN’s mission. The final change made to this variant was that it would not display the hero image to mobile users. Instead, the headline and CTA would be displayed on a simple colored background to make it easier to read on mobile devices.
Our hypothesis is that the additional clarity provided in Treatment #1 and Treatment #2 would cause more users to click the “GIVE NOW” button and go to the donation page. We theorized that Treatment #2 would perform the best in this experiment because it added the most clarity to the value proposition language and was designed to make that language easier to read for mobile users.
Research Question
We believe that adding increased clarity to the value proposition language around the homepage CTA will achieve an increase in clicks to the donation page by providing users with a better understanding of why they should make a gift and what their gift will actually do.
Design
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.27% | ||
| T1: | Treatment #1 | 0.44% | 59.8% | 95.4% |
| T2: | Treatment #2 | 0.58% | 111.3% | 99.9% |
This experiment has a required sample size of 5,343 in order to be valid. Since the experiment had a total sample size of 31,827, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
111.3% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that enhancing the value proposition language around the Call-to-Action (CTA) on the homepage had a significant impact on user engagement. The treatment variant, which added clarity, appeal, and credibility to the value proposition, resulted in a 111.3% increase in clicks to the donation page.
These findings suggest that users responded positively to the added information about how their gift will be used to provide humanitarian aid and share the Gospel. Additionally, the inclusion of body copy that highlighted the problem and reassured readers about the use of their donation likely increased trust and motivation to give.
In preparing for this experiment, one concern we had was that the length of variation two could create too much friction and that web visitors would instead prefer the shorter copy of version 1. The results however found the opposite. The longer copy provided even more details about the impact of a donor’s gift, leading to higher donor conversion. While running this experiment, we began testing these value propositions on the donation page as well. The visitors in those experiments were excluded from this one to keep the tests clean, but we will use those tests to further refine the value proposition provided for donating to CBN.
We attempted to dig into donor conversion on the main donation page, but had an issue with the UTMs being stripped on the traffic coming to the page from this CTA.