How adding an email field to a prayer request offer impacts conversion Experiment ID: #19976

Focus on the Family

Focus on the Family is a global Christian ministry dedicated to helping families thrive. We provide help and resources for couples to build healthy marriages that reflect God's design, and for parents to raise their children according to morals and values grounded in biblical principles.

Experiment Summary

Timeframe: 1/16/2020 - 2/21/2020

On the Missionary Oblates prayer request page, they offer people the opportunity to send in prayer requests. They see thousands of requests come in every week. Because of this and the engagement they see, they were curious if they could acquire email addresses for these people without discouraging people from sending in prayer requests. They added the email field as an option on the page and test this against the control.

Research Question

What is the impact on prayer request conversion when you add an email field to the page?

Design

C: Without email field
T1: With email field

Results

Treatment Name Conv. Rate Relative Difference Confidence
C: Without email field 59.4%
T1: With email field 55.7% -6.3% 99.5%

This experiment has a required sample size of 1,356 in order to be valid. Since the experiment had a total sample size of 5,479, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 6.3% decrease in conversion rate
× 0% increase in average gift

Key Learnings

We saw a 6% decrease in prayer requests when the email field was presented as an optional form field. That said, we acquired 497 email addresses from the treatment with 16% of them new. Even though we saw the minimal decrease in prayer requests, the value from the increased engagement of people already on our file and the acquisition of new emails is a greater benefit in the long run than the 114 prayer requests that didn’t get submitted because of the additional optional field.


Experiment Documented by...

Courtney Gaines

Courtney is a Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.