How prequalifying stickybar audiences to a campaign impacts donor conversion rate
The Fund for American Studies
Experiment Summary
Ended On: 02/25/2021
The Fund for American Studies was running on-site stickybar and home page popup promotions for their Founders’ Day Campaign and wanted to experiment with whether “selling the click” or “selling the donation” would be most effective for their audience visiting the website.
We decided to call these treatments “shopper” language (to sell the click) and “buyer” language (to sell the post-click donation).
Research Question
We believe that buyer language used on-site to promote a campaign for web visitors will achieve increase donor conversion rate.
Design
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Shopper Language | 0.16% | ||
| T1: | Buyer Language | 0.39% | 141.7% | 81.1% |
This experiment has a required sample size of 4,030 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.
Key Learnings
With the deadline approaching for the campaign, we decided to take the version of the stickybar/popups that was showing a lead for the remaining few days.
We did observe some other statistics that are worth noting:
- Ad clickthrough rate on the “buyer” version of the ad copy did see valid decrease during the valuation period — showing 39.1% less clicks (LoC: 99%).
- However, because the fewer people clicking on the “buyer” version of the ad did convert (as shown above) at a higher rate, we observed an increase in donor conversion rate after they clicked the on-site ads.
- Combining more donors (as noted previously), the “buyer” version of the ad produced a more generous donor to the campaign — showing an increase of +18.4% increase in the size of the average gift.
- Because of that higher donor total and a higher average gift, the revenue for the “buyer” ads actually was a lift of +183% (LoC: 93.6%).
For this reason, we decided to end the experiment early and focus on getting the most returns on the effort for the remaining few days of the campaign.
In the future, it would appear that pre-qualifying the audience with donation appeals in the on-site promotions using stickybars and popups is advised for TFAS.