How adding a continuation headline and personalizing the body copy to come from a prominent figure impacted donations
American Cornerstone Institute
Experiment Summary
Timeframe: 04/25/2023 - 05/21/2023
ACI’s Cancel Culture Pledge funnel is producing a low instant donor conversion rate. The entry rate and sign up rate are above benchmark for the offer, which presents an opportunity to optimize the offer at the donation page level to see if we can improve our donor conversion rate.
Evaluating the current donation page, what first stood out is the headline. By simply thanking someone for signing up for the pledge, users may assume that they’ve reached the end of the engagement. Most consumer transactions conclude with a thank you, but we want visitors to stick around to engage with the value proposition copy and make a donation.
That’s why we developed a treatment variant (treatment #1) with the following headline: “Your name has been added. Before you go, would you consider this special request from Dr. Ben Carson?”
Through this headline, visitors will know that something important is waiting for them, and they should therefore continue on in their engagement, which is exactly what we want for them to do.
We then transitioned to the body copy and wondered if the value proposition message would resonate more strongly with potential donors if it was presented in the first person. We felt particularly confident in this assessment given the notoriety of ACI’s Founder.
Thus, we developed treatment #2, reframing the body copy to be presented from the 1st person point of view. We kept the copy consistent with it’s previous version, only adjusting the point of view. In addition, we added the signature of the messenger to add an even more personal touch.
Let’s see how each treatment performed.
Research Question
We believe that adding a continuation headline matched with personalizing the donation value prop copy for potential donors will achieve an increase in donations because more visitors will engage with the value prop and will be more motivated to give due to the personal message coming from the organization’s founder.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | Control | 2.1% | $0.00 | ||
| T1: | Headline | 4.1% | 95.4% | 93.3% | $0.00 |
| T2: | Headline + Body | 4.5% | 114.0% | 96.6% | $0.00 |
This experiment has a required sample size of 677 in order to be valid. Since the experiment had a total sample size of 1,484, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 114.0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learnings from this experiment are that adding a continuation headline and personalizing the donation value proposition copy for potential donors can significantly increase donations.
This combination resulted in a 114% increase in donor conversion rate with a valid 96.6% level of confidence. It outperformed the “headline only” treatment, which resulted in a 95.4% increase in donations with a nearly valid 93.2% level of confidence.
Treatment #2 was especially effective among new visitors, which produced a valid 322.6% increase in donations. This treatment also produce a valid lift in revenue.
The continuation headline led more visitors to the value proposition copy, and the personalized message within created a stronger emotional connection with potential donors, motivating them to engage with the value proposition and ultimately donate.
In future experimentation, we should continue to test the impact of personalized messaging and content on donor conversion rates. We can also experiment with different ideas for continuation headlines to find the most effective approach for engaging visitors and encouraging them to donate. Additionally, we can test the impact of personalization on other key metrics, such as user engagement and retention.