Luther Seminary

How lowering the gift array options for an acquisition offer affects donor conversion rate

Experiment ID: #108030

Luther Seminary

Experiment Summary

Timeframe: 09/26/2022 - 11/19/2022

Luther Seminary has been running an acquisition offer on Facebook for a free study guide for the book of Revelation. We previously tested the concept of lowering the gift array on the instant donation page for acquisition offers, so we wanted to further test this concept for this offer as well.

As you can see in the images below, the gift array options ranged wide and high for an offer with perceived low value that was ultimately being presented to new names and potentially new donors.

Research Question

We believe that lowering the gift array options for people who downloaded a study guide will achieve a higher donor conversion rate on the instant donation page.

Design

C: Control
T1: Lower gift array

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 2.8% $0.00
T1: Lower gift array 5.3% 88.5% 99.9% $0.00

This experiment has a required sample size of 483 in order to be valid. Since the experiment had a total sample size of 2,757, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 88.5% increase in conversion rate
× 0% increase in average gift

Key Learnings

The lower gift array resulted in an 88.5% increase in donor conversion rate with a 99.9% LoC. We believe this is because the value of the content offer received was not equal to the value of donation options given on the original donation page. The options presented in the treatment were far more tangible to someone who is brand new coming onto the Luther Seminary file as a majority of our samples were new site visitors.

Another notable takeaway is how the lower gift array affected overall revenue. We saw a 52% increase in revenue with a 96% level of confidence on the treatment page because the lift in conversion rate was so high. The lower gift array did see a lower average gift of about $6, but overall, that does not affect our revenue from this offer.

We believe this is because of how people view the level of complexity and perceived value each offer (and specifically the book of the Bible) provides to the consumer.

Furthermore, we’d like to continue to test this concept on new acquisition offers to see how best that audience responds.

Notable changes to other metrics due to the experiment:

  • Revenue experienced a 52.0% increase with a 96% level of confidence.


Experiment Documented by NextAfter

Question about experiment #108030

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.