The Gospel Coalition

How adding clarity and tangibility to monthly giving premiums impacted donations

Experiment ID: #114618

The Gospel Coalition

Experiment Summary

Timeframe: 11/09/2022 - 12/23/2022

A ministry client incentivizes recurring giving by offering two premiums on their recurring giving donation page. When a recurring gift is made, the donor receives a copy of two different eBooks. With the recurring giving page not producing donations at the rate we hoped for, we put together an experiment.

We wondered if the lack of conversions is in part a result of limited clarity regarding the premiums. The eBook premiums are called out on the page by title, but the title alone does not tell the page visitor what the eBook is about, nor does it show them what the eBook looks like. A description should give page visitors clarity, connection, and application – they should know how the premium can help improve their life and their faith. Plus, imagery is powerful, and showing an image of each eBook could be appealing and a draw to the page visitor.

Thus, we developed a treatment: When the eBook premium title is clicked, a popup appears displaying the eBook cover image, title, and CTA telling the visitor how the eBook can help improve their faith or be applied to their faith.

We’ll see if the added clarity and tangibility improve donation conversions, specifically recurring ones.

Research Question

We believe that adding clarity and tangibility to monthly giving premiums for page visitors will achieve an increase in recurring donations.

Design

C: Control
T1: eBook Offers Pop-up

Results

 Treatment NameConv. RateRelative DifferenceConfidenceAverage Gift
C: Control 4.5%$0.00
T1: eBook Offers Pop-up 16.1%253.8% 99.9%$0.00

This experiment has a required sample size of 52 in order to be valid. Since the experiment had a total sample size of 297, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 253.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

After running for more than a month, the treatment resulted in a valid 253.8% increase in donations. Oddly enough, only one of the donations the treatment produced was recurring. In fact, the control produce two recurring gifts to the treatment’s one.

What might this mean? It’s possible that page visitors thought they would receive the premiums when making a gift of any type, not just recurring. It does however clearly say in the “SPECIAL OFFER” section of the copy that the premiums would only be sent upon someone making a recurring donation.

It is also possible that overall giving increased to such a degree because visitors saw the type of offers TGC generates and wanted to do their part to support that work. Perhaps the popups gave credibility to the organization through proof of work, which many donation pages don’t make clear.

Despite not seeing a lift in recurring donations, the number of donations overall produced by the treatment, and valid lift, led us to roll the treatment out to 100% of traffic.

Notable changes to other metrics due to the experiment:

  • Revenue experienced a 111.2% increase with a 89% level of confidence.

Observations of the impact on other visitor segments in the experiment:

  • Desktop visitors had a 320.9% increase in donations with a 100% level of confidence.
  • New visitors had a 584.6% increase in donations with a 96% level of confidence.
  • Returning visitors had a 200.8% increase in donations with a 99% level of confidence.


Experiment Documented by NextAfter

Question about experiment #114618

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.