How a Wiki-Style Dear Reader impacted donor conversion
Experiment Summary
Timeframe: 04/24/2023 - 04/30/2023
Hour of Power is a Christian broadcast ministry that shares inspiring sermons, ebooks, and daily devotionals through their media presence and website. They have seen success in increasing email acquisition on their website’s daily devotional pages by adding Dear Readers below the devotion, and for their spring high-urgency campaign we decided to see how their traditional Dear Reader asking readers to become sustaining monthly donors would perform against a Wiki-Style Dear Reader.
The traditional Dear Reader appears on devotional pages underneath the devotion. The Wiki-Style Dear Reader appears closer to the top of the page and is above the devotional content. If a user scrolls down the page and goes past it, the Wiki-Style Dear Reader will disappear and a smaller version of it will appear at the bottom of the screen like a sticky bar.
The traditional Dear Reader only has one CTA button which brings the user directly to the campaign’s donation page, but the Wiki-Style version has two CTA buttons. The first button also leads the user directly to the donation page, but the second button that says “Give a Different Amount” opens up a small gift array within the Wiki-Style Dear Reader. The user can then select or enter the amount they would like to give and then they will be redirected to the donation page with their selection automatically chosen.
Our hypothesis was that new placement and style of the Wiki-Style Dear Reader would be more arresting and eye-catching for users on the page and would thus increase donor conversion.
Research Question
We believe that a Wiki-Style Dear Reader on Hour of Power‘s daily devotional pages will achieve an increase in donor conversion because it is more visually arresting and transforms into a sticky bar if the user scrolls past it.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Traditional Dear Reader | 1.4% | ||
| T1: | Wiki-Style Dear Reader | 1.1% | -24.7% | 90.5% |
This experiment has a required sample size of 7,629 in order to be valid. Since the experiment had a total sample size of 11,056, and the level of confidence is not above 95% the experiment results are not valid.
Key Learnings
This experiment did not fully validate, but we saw a decrease in donor conversion on the Wiki-Style Dear Reader with a 90% LOC. What this reveals about Hour of Power’s audience is that they do not respond as well to arresting popups/sticky bars that use high-urgency language. Instead, it shows us that their audience is more responsive to CTAs that take a conversational approach.
In future testing we can try using the same kind of conversational language in the Wiki-Style that we use in the Dear Reader to clarify if their audience wasn’t responding because of the language used in the Wiki-Style or if it was due the placement/presentation of the Wiki-Style.