How a revised value proposition and removal of a high-value premium offer affects donor conversion on an instant donation page Experiment ID: #11715

Dallas Theological Seminary

The DTS mission is, “to glorify God by equipping godly servant-leaders for the proclamation of His Word and the building up of the body of Christ worldwide.” They strive to help men and women fulfill the Great Commission and the Great Commandment, or more simply: Teach Truth. Love Well.

Experiment Summary

Timeframe: 1/29/2019 - 2/15/2019

DTS has been running their Genesis online course for some time now, but experienced a lower instant donor conversion after a couple of months of running it. They wanted to test the effectiveness of some copy (#6157) that was previously run on the Revelation course, along with the removal of a high-dollar premium offer that ran on the page. 

Research Question

Does a revised value proposition and the removal of a high-dollar premium offer affect donor conversion on an instant donation page?

Design

C: Control
T1: Donation focused copy treatment, no premium

Results

Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.75% $0.00
T1: Donation focused copy treatment, no premium 2.6% 248.8% 97.0% $0.00

This experiment has a required sample size of 381 in order to be valid. Since the experiment had a total sample size of 863, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 248.8% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment version of the copy, along with the removal of the high dollar premium, increased donor conversion rate by 248%. This copy approach leads in more quickly into the donation ask and value proposition than the control, which focuses more on the next steps for the actual course, rather than the big picture value of these online courses to believers everywhere. This approach, along with the removal of a high-dollar premium, let to significantly more donors converting on the page. 


Experiment Documented by...

Allison Jones

Allison is an Optimization Associate at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.