How increasing interest through an additional step in an acquisition funnel affects donor conversion Experiment ID: #36467

Bill of Rights Institute

Experiment Summary

Timeframe: 7/1/2020 - 7/29/2020

BRI was running an offer for a quiz about Natural Rights. They also were running an offer for an eBook about Natural Rights. So we combined the two into a treatment; they were first offered the quiz and when they received their score they were offered the eBook. This did mean that we asked them to go through one more step and would potentially lose new names. However, we wanted to see if increasing the value of the offer through adding a quiz would increase donor conversion.

Research Question

Does adding a quiz to an eBook offer add more value and in turn convert more donors?

Design

C: eBook-Only Offer
T1: Quiz-to-eBook Offer

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: eBook-Only Offer 0.01%
T1: Quiz-to-eBook Offer 0.04% 208.8% 100.0%

This experiment has a required sample size of 24,438 in order to be valid. Since the experiment had a total sample size of 247,617, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 208.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

Adding the quiz to the beginning of the eBook funnel resulted in a 208% increase in donor conversion. The treatment had a lower average gift with resulted in a net 140% increase in revenue (with a 99% LoC). And because Facebook had to reach fewer people to convert them into donors, the ROI on the Quiz-to-eBook funnel was 42% higher.

We expected the Quiz-to-eBook funnel to reduce email conversion because it added an extra step and people would likely decide not to continue. Surprisingly, the extra step actually increased email acquisition by 51%.

Overall, the results show that adding a quiz to an eBook offer can increase email and donor conversion. We think this is true for a few reasons. First, beginning with a quiz invites the audience to move beyond their presuppositions and piques their interest in the content of the eBook and the mission of the organization. Second, because of the addition of the quiz, the audience is engaged longer with the content and receives added value. Because of this they are more engaged with the topic and the organization, and so their relationship with the mission grows and their generosity blossoms.


Experiment Documented by...

Gina Gallutia

Gina is . If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.